Techniques for using in-air hand gestures detected via a wrist-wearable device to operate a camera of another device, and wearable devices and systems for performing those techniques

ABSTRACT

Systems and methods are provided for using a gesture detected at a wrist-wearable device to cause presentation of camera data at a head-wearable device. One example method includes receiving, via one or more sensors of a wrist-wearable device worn by a user, data generated based on performance of an in-air hand gesture by the user. In accordance with a determination that the data indicates that the in-air hand gesture is a camera-control gesture, the method further includes causing presentation to the user, via a display of a head-wearable device that is worn by the user and that is in communication with the wrist-wearable device, of a representation of camera data from a camera that is in communication with one or both of the wrist-wearable device and the head-wearable device.

RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 63/414,886, filed Oct. 10, 2022, titled “Techniques For Using In-AirHand Gestures Detected Via A Wrist-Wearable Device To Operate A CameraOf Another Device, And Wearable Devices and Systems For Performing ThoseTechniques,” and U.S. Provisional Patent Application No. 63/337,893,filed May 3, 2022, titled “Techniques for Using In-Air Hand GesturesDetected Via A Wrist-Wearable Device To Operate A Camera Of AnotherDevice, And Wearable Devices and Systems For Performing ThoseTechniques,” which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to wearable devices and methodsfor enabling quick and efficient capture of camera data and thepresentation of a representation of the camera data at a coupleddisplay, more particularly, to wrist-wearable devices configured tooffer low-friction camera-control gestures based on the user'sperformance of a hand gesture and cause the capture of camera data atanother device (e.g., a head-wearable device) and the presentation ofthe representation of the camera data at a display of another device(e.g., the head-wearable device).

BACKGROUND

Computing devices, such as computers, phones, and tablets, require usersto navigate through a number of different user interfaces and/or providemultiple inputs to capture camera data and present the captured data.Requiring multi-step input processes through different applications andthen through different user interfaces within those differentapplications requires a user to stop performing an activity (removingthemselves from an experience) to interact with the computing devicesfor an extended time period, thereby wasting time and creating userdissatisfaction with such inefficient use cases. Additionally, requiringmulti-step input processes for sharing data (such as an image or video)via computing devices requires the devices to be powered on longer andconsume their available energy, thereby depleting what can be a limitedpower supply. Techniques for users to capture pictures and share themwith their contacts are inadequate as they require direct userintervention at multiple points in the process (e.g., a user has toselect activate an application, perform an action at the application,request to share content from the application, locate a contact theywish to share content with, select and share the content with thecontact), thereby becoming burdensome to users, while also inefficientlymaking use of limited computing and power resources for certaincomputing devices (particularly wrist-wearable devices).

Many actions to control cameras are also not low friction as they caninvolve large arm movements, extended movements of a device to capture adesired image, and/or noisy voice commands that are unsuitable incertain social circumstances.

As such, there is a need for addressing one or more of the drawbacksdiscussed above by developing techniques to enable quick and efficientmethods for a user to capture, present, and/or share camera data usinglow-friction inputs detected by a sensor of a wrist-wearable device.

SUMMARY

The wrist-wearable devices, head-wearable devices, and methods of usethereof (as well as systems including both wrist-wearable andhead-wearable devices) described herein address one or more of theabove-mentioned drawbacks by seamlessly allowing a user to interact withimaging devices using a wrist-wearable device. In particular, thewrist-wearable device can cause one or coupled devices, such as ahead-wearable device, to seamlessly perform one or more actionsassociated with operating an imaging device and/or applications at thewrist-wearable device and/or the head-wearable devices. As one example,a method of using a gesture detected at a wrist-wearable device topresent camera data at a head-wearable device is provided. Thewrist-wearable device receives, via one or more sensors on thewrist-wearable device worn by a user, data based on performance of anin-air hand gesture by the user, and in accordance with a determinationthat the data indicates that the in-air hand gesture is a camera-controlgesture, causes the presentation, via a display of a head-wearabledevice that is worn by the user and that is in communication with thewrist-wearable device, a representation of camera data from a camerathat is in communication with one or both of the wrist wearable deviceand the head-wearable device.

This can be done without requiring the user to perform a lengthysequence of inputs, instead, in some embodiments, the number of userinputs (e.g., pinch gestures detected by a wrist-wearable device) can beas few as one (one user input to cause activation of the imaging deviceand then capturing of an image using the imaging device). Further, thewrist-wearable devices described herein improve users' daily activitiesand productivity by reducing the number of direct inputs required by auser to access an application and/or capture images. For example, thewrist-wearable devices and methods described herein, in one embodiment,provide improved techniques for quickly capturing images and sharingthem with contacts. In particular, a user, in some embodiments, canshare images via image-sharing applications, with just one or twogestures detected by a wrist-wearable device. The reduced number ofrequired inputs do not exhaust power and processing resources of awrist-wearable device, thereby extending the battery life of thewrist-wearable device while further sustained user interactions with newsystems make use of a wrist-wearable device as an input controller usedto operate hardware at another device (e.g., in-air hand gesturesdetected by sensors of a wrist-wearable device to operate ahead-wearable device's camera, all of which can be done usinglow-friction gestures in which a user need only move their digits andneed not make large arm movements that would be socially unacceptable).

To help further the above goals, and as was briefly noted above, someembodiments described herein also make use of components of otherwearable devices, such as a head-wearable device (e.g., smart glasses orvirtual reality headset), including speaker, microphone, and/or cameraof these other wearable devices to allow for seamlessly capturing andsharing data between the user and others. For instance, in someembodiments, a user can request, via the wrist-wearable device, tocapture and share an image with a contact, and the wrist-wearabledevice, responsive to the request, can cause an imaging device of ahead-wearable device to activate and be used in the capture of an imageto be shared with the contact. Determinations can also be conducted asto whether an imaging device of the wrist-wearable device or thehead-wearable device should be utilized (e.g., based on current powerlevels at each of the devices and/or based on available computingresources at each of the devices, and the like), and the appropriateimaging device can be activated based on those determinations, therebyfurthering the benefits of making efficient use of limited power andcomputing resources at wearable devices. Additional examples areexplained in further detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

A more particular description may be had by reference to the features ofvarious embodiments, some of which are illustrated in the appendeddrawings. The appended drawings illustrate pertinent example features ofthe present disclosure. The description may admit to other effectivefeatures as the person of skill in this art will appreciate upon readingthis disclosure.

FIGS. 1A-1F illustrate coordinating operations of a wrist-wearabledevice and a head-wearable device to cause presentation of camera data,in accordance with some embodiment.

FIG. 2A-2F illustrate different hand gestures performed by a user, inaccordance with some embodiments.

FIGS. 3A-3N illustrate adjustments to a representation of camera datausing one or more hand gestures performed by a user, in accordance withsome embodiments.

FIGS. 3O-3Z illustrate selection of one or more capture modes using oneor more hand gestures performed by a user, in accordance with someembodiments.

FIGS. 4A-4J illustrate different surface-contact gestures performed by auser, in accordance with some embodiments.

FIGS. 5A-5C illustrate coordination of a wrist-wearable device and ahead-wearable device to cause presentation of camera data in an ARenvironment, in accordance with some embodiments.

FIGS. 6A and 6B illustrates a flow diagram of a method for detecting anin-air hand gesture and determining if the in-air hand gesture is acamera-control gesture, according to some embodiments.

FIGS. 7A and 7B illustrates a detailed flow diagram of a method of usinga gesture detected at a wrist-wearable device to cause presentation ofcamera data at a head-wearable device, according to some embodiments.

FIG. 8A-8H illustrate a head-wearable device including a vestigialmuscle training mode for training a user to use their vestigial earmuscles to perform vestigial ear muscle gestures that are detectable bythe head-wearable device, according to some embodiments.

FIG. 9 illustrates a detailed flow diagram of a method of detectingvestigial ear movements at a head-wearable device, according to someembodiments.

FIG. 10 illustrates an example system for capturing and presentingcamera data, in accordance with some embodiments.

FIG. 11 illustrates a system of one or more devices for capturing andpresenting camera data, in accordance with some embodiments.

FIGS. 12A-12E illustrate an example wrist-wearable device, in accordancewith some embodiments.

FIGS. 13A-13B illustrate an example AR system in accordance with someembodiments.

In accordance with common practice, like reference numerals may be usedto denote like features throughout the specification and figures.

DETAILED DESCRIPTION

Numerous details are described herein to provide a thoroughunderstanding of the example embodiments illustrated in the accompanyingdrawings. However, some embodiments may be practiced without many of thespecific details, and the scope of the claims is only limited by thosefeatures and aspects specifically recited in the claims. Furthermore,well-known processes, components, and materials have not necessarilybeen described in exhaustive detail so as to avoid obscuring pertinentaspects of the embodiments described herein.

FIGS. 1A-1F illustrate coordinating operations of a wrist-wearabledevice 170 and a head-wearable device 110 to cause presentation ofcamera data, in accordance with some embodiments. In particular, usinggestures detected at a wrist-wearable device 170 to cause presentationof camera data at a head-wearable device 110. In some embodiments, thewrist-wearable device 170 is communicatively coupled with thehead-wearable device 110 (e.g., by way of a Bluetooth connection betweenthe two devices, and/or the two devices can also both be connected to anintermediary device such as a smartphone that provides instructions anddata to and between the two devices). The wrist-wearable device 170includes one or more displays 130 a (e.g., a touch screen 130 e) forpresenting a visual representation of data to a user 115, speakers forpresenting an audio representation of data to the user 115, microphonesfor capturing audio data, imaging devices 111 (e.g., a camera) forcapturing image data and/or video data (referred to as “camera data”),and sensors (e.g., sensors 1121, such as neuromuscular-signal sensors(e.g., electromyography (EMG) sensors), inertial measurement units(IMU)s, biometric sensors, position sensors, and/or any other sensorsdescribed below in reference to FIG. 11 ) for detecting and determiningone or more hand gestures performed by the user 115. The head-wearabledevice 110 includes one or more imaging devices 111, microphones,speakers, displays 130 (e.g., a heads-up display 130 b, a built-in orintegrated monitor or screen, a projector, and/or similar device),and/or sensors 1121. In some embodiments, the one or more components ofthe head-wearable device 110 described above are coupled with thehousing and/or lenses of the head-wearable device 110. In someembodiments, the one or more components of the wrist-wearable device 170described above are coupled with a wrist-wearable structure (e.g., aband portion) of the wrist-wearable device 170, housed within a capsuleportion of the wrist-wearable device 170 or a combination of thewrist-wearable structure and the capsule portion. In some embodiments,the wrist-wearable device 170 and the head-wearable device 110 arecommunicatively coupled via an intermediary device (e.g., a server 1170,a computer 1174 a, a smartphone 1174 b and/or other devices describedbelow in reference to FIG. 11 ) that is configured to control thewrist-wearable and head-wearable devices 170 and 110.

In some embodiments, the wrist-wearable device 170 is configured tocause the head-wearable device 110 (and/or other communicatively coupleddevice with an imaging device 111) to capture, receive, and/or presentcamera data. Particularly, in some embodiments the wrist-wearable device170 is configured to detect hand gestures (e.g., in-air hand gesture,surface-contact gestures) performed by the user, and if the hand gestureis determined as a camera-control gesture, cause the head-wearabledevice 110 to capture, receive, and/or present camera data. In someembodiments, the different hand gestures are detected using only thesensors of the wrist-wearable device 170. For example, a hand gesturecan be detected using only neuromuscular-signal data sensed byneuromuscular-signal sensors, IMU data sensed by IMU sensors, etc. Insome embodiments, the sensor data of the wrist-wearable device 170 canbe combined to detect that a hand gesture was performed. For example,the wrist-wearable device 170 can detect a hand gesture usingneuromuscular-signal data and IMU data. In some embodiments, the handgesture is identified as a camera-control gesture based, in part, onsensor data from one or more sensors of the wrist-wearable device 170,which is obtained while the wrist-wearable device 170 is worn by theuser 115. For example, the wrist-wearable device 170 can detect a handgesture and determine that the hand gesture is a camera-control gesturebased, in part, on EMG sensor data and/or IMU data detected by its oneor more sensors. Alternatively, or in addition, in some embodiments, thehand gesture is determined as a camera-control gesture based on cameradata captured by communicatively coupled imaging devices 111. Forexample, camera data captured by the head-wearable device 110,wrist-wearable device 170, and/or other imaging device communicativelycoupled with the wrist-wearable device 170 can be processed (e.g.,computer vision) to identify a hand gesture.

The hand gesture can be an in-air gesture, a surface-contact gesture,and or other gestures that can be detected and determined based onmovements of a single hand (e.g., a one-handed gesture performed with auser's hand that is wearing/donning the wrist-wearable device). In-airmeans, in some embodiments, that the user 115's hand 120 does notcontact a portion of the wrist-wearable device 170 (or othercommunicatively coupled device), in other words the gesture is performedin open air in 3D space and without contacting the wrist-wearable device170. Surface-contact gestures (contacts at surfaces other than thewrist-wearable device 170) more generally are also contemplated in whichneuromuscular signals sensed by one or more EMG sensors (or other typesof data from other sensors, such as proximity sensors, time-of-flightsensors, sensors of an inertial measurement unit, etc.) can be used todetect a contact (or an intention to contact) a surface (e.g., a singleor double finger tap on a table, on a user's leg, a couch, etc.). Thus,while the primary example herein is an in-air gesture, the disclosure isnot limited to those in-air gestures, as other gestures that do notcontact a wrist-wearable device 170 are also contemplated, including thesurface-contact gestures just described. Further, hand gestures can beassociated with one or more commands other than a camera-controlgesture. The hand gestures, when detected and determined as a respectivecommand by one or more processors (e.g., processors 1150; FIG. 11 ), areconfigured to cause an action to be performed at a computing device,such as the wrist-wearable device 170, head-wearable device 110, orother device described below in reference to FIG. 11 . In other words,although FIGS. 1A-1F illustrate the coordination between thewrist-wearable device 170 and the head-wearable device 110 usingdetected gestures by the wrist-wearable device 170, gestures detected bythe wrist-wearable device 170 can cause other devices (e.g.,smartphones, tablets, security cameras, etc.) to capture, provide,and/or present camera data as described below in reference to FIGS.4A-4J.

In some embodiments, the head-wearable device 110 is a pair of smartglasses, augmented reality (AR)goggles (with or without a heads-updisplay), AR glasses (with or without a heads-up display), otherhead-mounted displays, or head-wearable device 110). The head-wearabledevice 110 is configured to capture camera data via an imaging device111 and/or present a representation of the camera data via the display130. In some embodiments, the display 130 is coupled with one or both ofthe lenses of the head-wearable device 110. Camera data presented by thedisplay 130 is presented in conjunction with the field of view 134 ofthe user 115. In some embodiments, the camera data is overlayed over aportion of the field of view 134 of the user 115. In addition, in someembodiments, the head-wearable device 110 is configured to capture audiodata via a microphone and/or present a representation of the audio datavia speakers.

In some embodiments, head-wearable device 110 is configured to presentan artificial-reality environment (e.g., AR, virtual reality (VR)environment, and the like) and/or perform one or more commands in anartificial-reality environment. In some embodiments, the head-wearabledevice 110 can be an artificial-reality system including, but notlimited to, non-immersive, semi-immersive, and fully-immersive VRsystems; marker-based, markerless, location-based, and projection-basedaugmented reality systems; hybrid reality systems; and other types ofmixed reality systems.

Turning to FIG. 1A, the user 115 is viewing a physical environmentincluding one or more objects while wearing the head-wearable device 110and the wrist-wearable device 170. While wearing the wrist-wearabledevice 170, sensor data is monitored to sense a hand gesture performedby user 115. The sensor data can include EMG data (e.g., one or moreneuromuscular signals sensed by EMG sensors); IMU data (e.g., position,orientation, and/or other data sensed by one or more IMUs); accelerationdata (e.g., sensed by one or more accelerometers), as well as othersensor data described below in reference to FIG. 11 . The data used todetect and determine a hand gesture can also be data from an imagingdevice 111 that is able to see the user 115's hand 120 to use computervision to detect gestures. One or all of the sensors can be utilized toprovide data for detecting and determining a hand gesture. The sensedhand gestures are further determined as a particular control gesture(e.g., a camera-control gesture) based, in part, on the sensor data. Forexample, when the user 115 moves a portion of their hand or intends tomove a portion of their hand (e.g., phalanges 140, 150, etc.), thewrist-wearable device 170 detects position and/or orientation data aswell as neuromuscular signals generated by the user 115's hand 120movement. The wrist-wearable device 170, using the one or moreprocessors 1150, determines the type of hand movements performed orintended to be performed by the user 115 as well a control commandassociated with the hand gesture based on the detected position and/ororientation data and neuromuscular signals. As described below, thewrist-wearable device 170 provides one or more commands associated withthe hand gesture to another device, such as the head-wearable device110, to perform a corresponding action associated with operating theother device. Alternatively, in some embodiments, the wrist-wearabledevice 170 provides sensor data associated with the hand gesture toanother device such that the other device can determine the type of handgesture performed by the user 115.

FIG. 1B illustrates the field of view 134 of the user 115 in FIG. 1A. Inparticular, the user 115's view before any hand gesture is performed anddetected by the wrist-wearable device 170. The user 115 can move aroundfreely without having their field of view 134 obstructed. For example,as the user 115 moves their head the field of view 134 of the user 115changes based on the user 115's movements. As described above, in someembodiments, the display 130 is coupled with a lens or both lenses. Insome embodiments, the display 130 is positioned over a portion of thelens without obstructing the user 115's view. Alternatively, in someembodiments, the display 130 makes up all or a substantial portion ofthe lens or lenses (while still allowing the user 115 with fullvisibility of their field of view 134). In some embodiments, the display130 is a heads-up display coupled with or integrated into one or bothlenses, or a heads-up display coupled with or integrated with a portionof the frame around the lenses.

FIG. 1C illustrates the user 115 performing a camera-control gesturethat is detected by the wrist-wearable device 170. More specifically,the performance of the hand gesture generates sensor data that is sensedby one or more sensors of the wrist-wearable device 170, which is usedby the wrist-wearable device 170 to determine that the hand gesture is acamera-control gesture (e.g., a double tap of the phalanges 140 and150). A camera-control gesture is a hand gesture associated with one ormore commands or instructions configured to control an imaging device111. In some embodiments, the camera-control gesture is associated withinstructions to cause an imaging device 111 to capture camera data(e.g., continuously capture a video stream, capture a single image,capture a plurality of images, capture a sequence of images, etc.),adjust camera data (e.g., zoom in, zoom out, pan in different directions(e.g., left, right, up, down, and diagonally), etc.), switch betweendifferent camera modes (e.g., panoramic, video, high dynamic range (HDR)mode, night mode, etc.), and/or other camera operations.

In some embodiments, in accordance with a determination, by thewrist-wearable device 170 that the sensor data indicates that the handgesture is a camera-control gesture for capturing camera data, thewrist-wearable device 170 provides one or more commands or instructionsassociated with the camera-control gesture to another device includingor communicatively coupled with an imaging device 111, such as thehead-wearable device 110. The commands or instructions associated withthe camera-control gesture cause the imaging device 111 of the otherdevice to capture camera data. For example, when the double tap of thephalanges 140 and 150 is detected by the wrist-wearable device 170, thehead-wearable device 110 is caused to capture camera data (as shown bycaptured image 160; FIG. 1D). More specifically, the camera-controlgesture demonstrated in FIG. 1C is the user 115's thumb (phalange 140)and pointer finger (phalange 150) performing a double tap action whichultimately results in the capture of the camera data. The above exampleis non-limiting; the user 115 could perform a double tap or double pinchgesture with other phalanges and is not only limited to the thumb(phalange 140) and pointer finger (phalange 150) (e.g., could also be athumb (phalange 140) and middle finger, thumb (phalange 140) and ringfinger, thumb (phalange 140) and pinky, two other fingers, etc.). Insome embodiments, the camera-control gestures are defined by the user115 (e.g., user defined camera-control gestures) or automaticallydefined (e.g., predefined camera-control gestures) by the wrist-wearabledevice 170 or other devices described below in reference to FIG. 11 .The camera-control gestures can be defined using a head-wearable device110, a smartphone 1174 b, the wrist-wearable device 170, or otherintermediary device. For example, a user 115 could define acamera-control gesture for capturing camera data to be a single tapinstead of a double tap by any two phalanges. The wrist-wearable device170 causes the action associated with the camera-control gesture to beperformed immediately (in response to detection) to the hand gesture. Inother words, in response to a detected camera-control gesture noadditional input is required from the user 115 to trigger an operationsuch as the capture of camera data.

FIG. 1D illustrates the field of view 134 of the user 115 in FIG. 1C.More specifically, FIG. 1D shows the user 115's view after thecamera-control gesture (e.g., the double tap of the phalanges 140 and150) is detected by the wrist-wearable device 170. The wrist-wearabledevice 170, in response to determining that the hand gesture is acamera-control gesture for capturing camera data, causes thehead-wearable device 110 to capture camera data. In some embodiments,the camera-control gesture for capturing camera data also causes thehead-wearable device 110 to present a representation of the camera data(e.g., captured image 160). In some embodiments, the representation ofthe camera data is displayed over a portion of the user 115's field ofview 134. In some embodiments, the representation of the camera data ispresented semi-transparent such that the user 115's view is notsubstantially obstructed (e.g., transparency of at least 25% such thatthe user 115 can see through the representation of the camera data). Insome embodiments, the user 115 can adjust the transparency such that therepresentation of the camera data is fully transparent (e.g.,invisibly), fully opaque, or different gradients in between.

In some embodiments, the representation of the camera data is presentedwithin a user interface 165 displayed at the display 130 of thehead-wearable device 110. In some embodiments, the user interface 165includes one or more affordance for interacting with the representationof the camera data. For example, the one or more affordances include asave affordance 171, a share affordance 172, a messaging affordance 173,an editing affordance 174, and a contacts affordance 175. The operationsperformed by the different affordance are discussed below in referenceto FIGS. 3A-3J. Similar to the representation of the camera data, theuser interface 165 is displayed over a portion of the user 115's fieldof view 134 and/or presented semi-transparent such that the user 115'sview is not substantially obstructed.

FIG. 1E illustrates the user 115 performing a camera-control gesture viaone or more vestigial ear muscle gestures (e.g., ear 810 movement) thatare detected by sensors (e.g., surface EMGs) coupled with thehead-wearable device 110. More specifically, the performance of avestigial ear muscle gesture generates sensor data that is sensed by oneor more sensors of the head-wearable device 110 and used by thehead-wearable device 110 to determine that the vestigial ear musclegesture was performed. The head-wearable device 110, in accordance witha determination that the sensor data indicates that the vestigial earmuscle gesture was performed, determines a control action, such as acamera-control gesture, associated with the vestigial ear muscle gestureand performs the control action or provides one or more commands orinstructions associated with the control action to cause another devicecommunicatively coupled with the head-wearable device 110 to perform thecontrol action. For example, when the movement of the vestigial earmuscle (e.g., the user 115 wiggling their ear 810) is detected by thehead-wearable device 110, the head-wearable device 110 determines avestigial ear muscle gesture associated with the movement of thevestigial ear muscle and, in accordance with a determination that thevestigial ear muscle gesture corresponds to a camera-control gesture(e.g., capture image data), the head-wearable device 110 captures imagedata via a communicatively coupled imaging device 111 (as shown bycaptured image 160; FIG. 1F).

FIG. 1F illustrates the field of view 134 of the user 115 in FIG. 1E.More specifically, FIG. 1F shows the user 115's view after the vestigialear muscle gesture corresponding to the camera-control gesture isdetected by the head-wearable device 110. In some embodiments, afterimage data is captured by performance of the vestigial ear musclegesture, the head-wearable device 110 present a representation of thecaptured image data to the user. The above examples of FIGS. 1E-1F arenon-limiting; the user 115 could move vestigial ear muscles of each earor both ears to cause the performance of one or more control actions,such as performing a call, initiating a digital assistant, initiatingone or more applications, navigating a user interface, etc. Vestigialear muscle gestures can include wiggling one or both ears; pinning oneor both ears backwards; moving one or both ears up, down, left, right; aparticular sequence or pattern of ear movements (e.g., moving the leftear twice, moving the left ear once and the right ear twice, moving bothears at the same time, etc.). As described below in reference to FIGS.8A-8H, the user 115 can be trained to use one or more vestigial earmuscles to perform different vestigial ear muscle gestures.

Although the examples of FIGS. 1A-1F describe the interaction betweenthe wrist-wearable device 170 and the head-wearable device 110, handgestures detected by the wrist-wearable device 170 can cause othercommunicatively coupled device to capture camera data, present cameradata, and/or provide captured camera data for presentation atcommunicatively coupled display.

FIG. 2A-2F illustrate different hand gestures performed by a user, inaccordance with some embodiments. In particular, hand gestures that aredetermined by a wrist-wearable device 170 to be different camera-controlgestures. As described above in reference to FIGS. 1A-1F, hand gesturesare determined to be one or more camera control-gestures based on sensordata obtained by one or more sensors 1121 (FIG. 11 ) of thewrist-wearable device 170.

In FIG. 2A, the user 115 performs and maintains a hand gesture (e.g.,pinching and holding phalanges 140 and 150 together, a pinch and holdgesture 251). In some embodiments, maintaining a hand gesture caninclude holding a phalange or phalanges still for a predetermined periodof time (e.g., two or more seconds) and/or holding at least twophalanges in contact for a predetermined period of time (e.g., two ormore seconds). Alternatively or additionally, in some embodiments,maintaining a hand gesture can include applying a predeterminedthreshold pressure between at least two phalanges or at least onephalange and a surface (e.g., a table, wall, palm, leg, etc.) for apredetermined period of time (e.g., at least two seconds). The handgesture performed by the user 115 generates sensor data (e.g., EMG data,IMU data, etc.) that is used by the wrist-wearable device 170 todetermine, using the one or more processors 1150 (FIG. 11 ), a cameracontrol-gesture.

In FIG. 2A, the pinch and hold gesture 251 is associated with a cameracontrol-gesture for displaying a preview of a representation of cameradata captured by an imaging device 111. The camera control-gesture fordisplaying a preview of a representation of camera data captured by animaging device 111, when detected, causes the head-wearable device 110to present, via a display 130, a representation of the camera datacaptured by an imaging device 111.

FIG. 2B illustrates a field of view 134 of the user 115 in FIG. 2A. Insome embodiments, in accordance with a determination the hand gesture isa camera control-gesture for displaying a preview of a representation ofcamera data captured by an imaging device 111 (e.g., the pinch and holdgesture 251), the head-wearable device 110 presents a preview 260 to theuser 115. The preview 260 includes a representation of camera datacaptured within a field of view 134 of the user 115. In someembodiments, the preview 260 includes a subset of the field of view 134or of the entire field of view 134 of the user 115. The preview 260 canbe a live feed (e.g., simultaneous capture and presentation) of what thecamera is viewing while the hand gesture is maintained. Alternatively,the preview 260 can be a recording of a field of view 134 captured bythe user 115. In some embodiments, a plurality of previews from one ormore cameras can be displayed on the head-wearable device 110. In someembodiments, the preview 260 is presented to the user 115 via a userinterface 165 as described above in reference to FIGS. 1A-1F.

In some embodiments, the head-wearable device 110 presents to the user115 one or more indications that notify the user 115 that the imagingdevice 111 is active (e.g., glasses icon 231), the microphone is active(e.g., mic icon 232), and/or the speakers are audible (e.g., speakericon 233). In some embodiments, the indications are provided to the user115 as long as the camera control-gesture for displaying a preview of arepresentation of camera data is detected. In some embodiments,respective indications are provided to the user when the device isactive.

FIG. 2C illustrates adjustments to the camera data while the user 115performs and maintains a hand gesture. In particular, while the user 115performs the pinch and hold gesture 251 (e.g., the cameracontrol-gesture for displaying a preview of a representation of cameradata), the camera data captured by the imaging device 111 ofhead-wearable device updates in accordance with changes to the user115's field of view 134. For example, while the user 115 performs andmaintains the pinch and hold gesture 251, the imaging device 111captured camera data reflecting changes in the user 115's head movement(e.g., moving side-to-side up and down, etc.). Alternatively or inaddition, in some embodiments, while the user 115 performs and maintainsthe pinch and hold gesture 251, movement of the user 115's wrist can beinterpreted to cause changes in the field of view 134 of the camera(e.g., panning the camera side-to-side, up and down, etc.).

FIG. 2D illustrates a field of view 134 of the user 115 in FIG. 2C. Inparticular, in accordance with a determination the pinch and holdgesture 251 is performed and maintained, the preview 260 presented via adisplay 130 of the head-wearable device is updated to show arepresentation of the camera data reflective of the user 115's field ofview 134 as they move their head. For example, as the user 115 movestheir head down, the preview 260 is updated to show the user's change infield of view 134 (e.g., the bottom of the object as opposed to thetop). The changes to the preview 260 are presented to the user while thepinch and hold gesture 251 is performed and maintained. In this way, theuser 115 has a visual representation of the camera data that is beingcaptured or about to be captured by the imaging device 111 of thehead-wearable device 110.

FIG. 2E illustrates the user 115 no longer performing and maintaining ahand gesture (e.g., releasing the hand gesture). Releasing a handgesture can include moving a phalange that was held still for apredetermined period of time; separating at least two phalanges 140 and150 that were contacting one another such that they are no longer incontact; moving a phalange or phalanges contacting a surface such thatthe phalange or phalanges are no longer contacting the surface; and/orremoving (e.g., no longer applying) a predetermined threshold pressurebetween at least two phalanges 140 and 150, or at least one phalange anda surface. The wrist-wearable device 170 determines that the user 115releases the hand gesture (e.g., pinch and hold gesture 251) based onsensor data obtained by the one or more sensors 1121. In someembodiments, when the user 115 releases the hand gesture, the cameracontrol-gesture associated with the hand gesture is completed or afollow-on action is performed. For example, releasing the pinch and holdgesture 251 can cause the head-wearable device to capture camera data,via an imaging device of the head-wearable device, of the user 115'sfield of view when the pinch and hold gesture 251 is released. In someembodiments, a representation of the captured image data is presented tothe user 115 via the display 130 of the head-wearable device 110.

FIG. 2F illustrates a field of view 134 of the user 115 in FIG. 2E.Specifically, the head-wearable device 110 presents, via display 130, afield of view of the user 115 when the pinch and hold gesture 251 isreleased. As described above, in some embodiments, when the pinch andhold gesture 251 is released, the imaging device 111 of thehead-wearable device captures camera data of the user 115's field ofview. Further, in some embodiments, when the pinch and hold gesture 251is released, a representation of the captured camera data is presentedto the user 115.

As described above in reference to FIG. 1D, the representation of thecamera data (e.g., captured image 160) is presented within a userinterface 165 displayed at the display 130 of the head-wearable device110. In some embodiments, the representation of the camera data ispresented at a display 130 of the wrist-wearable device 170. In someembodiments, the representation of the camera data is presented at thewrist-wearable device 170 based on a determination that the user 115 isfocused on the wrist-wearable device 170. A determination that the useris focused on the wrist-wearable device 170 is based on the by thesensor data obtained by the one or more sensors 1121 indicating that theuser 115 is looking at the wrist-wearable device 170.

FIGS. 3A-3N illustrate adjustments to a representation of camera datausing one or more hand gestures performed by a user, in accordance withsome embodiments. In particular, hand gestures that, when detected by awrist-wearable device 170, cause adjustments to camera data captured byan imaging device 111. As described above in reference to FIGS. 1A-1F,hand gestures are determined to be one or more camera control-gesturesbased on sensor data obtained by one or more sensors 1121 (FIG. 11 ) ofthe wrist-wearable device 170.

In FIG. 3A, the pinch and hold gesture 251 is associated with a cameracontrol-gesture for displaying a preview of a representation of cameradata captured by an imaging device 111. The camera control-gesture fordisplaying a preview of a representation of camera data captured by animaging device 111 (e.g., of head-wearable device 110), when detected,causes the head-wearable device 110 to present, via a display 130, arepresentation of the camera data captured by an imaging device 111. Asdescribed below, in some embodiments, multiple hand gestured can belinked together to perform one or more action sequentially orconcurrently. For example, additional hand gestures can be detected bythe wrist-wearable device 170 that cause one or more adjustments to thecamera data captured by an imaging device 111. Additionally, in someembodiments, the user is preparing to perform an additional hand gesturesuch as rotating the wrist 382, which starts at an angle theta.

FIG. 3B illustrates a field of view 134 of the user 115 in FIG. 3A. Insome embodiments, in accordance with a determination the hand gesture isa camera control-gesture for displaying a preview of a representation ofcamera data captured by an imaging device 111 (e.g., the pinch and holdgesture 251), the head-wearable device 110 presents, via a display 130,a preview 260 to the user 115. Similar to FIGS. 2A-2F, the preview 260includes a representation of camera data captured within a field of view134 of the user 115. In some embodiments, the preview 260 includesinformation corresponding to the camera data captured by the imagingdevice 111. For example, the preview 260 incudes a current magnificationof the representation of the camera data (e.g., zoom at 100%). In someembodiments, the information includes a capture time, capture location,one or more applied filter, camera modes, tags, stickers, and/or otherinformation related to the captured camera data. In some embodiments,the information notifies the user that representation of the camera datais an unaltered representation of the image. In some embodiments, whilecamera control-gesture for displaying a preview of a representation ofcamera data captured is maintained, the head-wearable device 110presents to the user 115 one or more indications that notify the user115 that the head-wearable device 110 is capturing imaging data (e.g.,glasses icon 231), capturing audio (e.g., mic icon 232), and/or thepresenting audio data (e.g., speaker icon 233).

FIG. 3C illustrates adjustments to the camera data while the user 115performs and maintains a hand gesture and performs an additionalcamera-controlling gesture. In particular, while the user 115 performsthe pinch and hold gesture 251 (e.g., the camera control-gesture fordisplaying a preview of a representation of camera data), the userperforms an additional gesture (e.g., rotating their wrist 382 by apredetermined degree) that causes the camera data captured by theimaging device 111 of head-wearable device to update. In someembodiments, rotation of a user 115's wrist is associated with acamera-control gesture for zooming-in or zooming-out that, when detectedby the wrist-wearable device 170, adjust the magnification of the cameradata captured by the imaging device accordingly.

FIG. 3D illustrates a field of view 134 of the user 115 in FIG. 3C. Inparticular, in accordance with a determination the pinch and holdgesture 251 is performed and maintained, and an additional cameracontrolling gesture is performed (e.g., the wrist 382 rotation), thepreview 260 presented via a display 130 of the head-wearable device isupdated to show a zoomed in representation of the camera data. Forexample, when the user 115 rotates their wrist, the preview 260 isupdated to show the user's field of view 134 magnified by 400% (e.g., acloser zoomed in view of the object). In other words, changes to themagnification of the representation of the camera data are presented tothe user 115 as the gesture is performed. In this way, the user 115 canvisually see the changes being performed on the camera data.

The amount of magnification applied to the representation of the cameradata is based on the rotation of the user 115's wrist. For example, themore the user rotates their wrist 382 counter-clockwise the greater themagnification. The user 115 can also control the rate at which themagnification is applied based on how fast the user 115 moves theirwrist 382. The different movements of the user 115's wrist 382 aredetected by the one or more sensors of the wrist-wearable device 170 andused to determine how the camera-control gestures (e.g., themagnification) should be applied. In some embodiments, the user 115holds their wrist 382 in a rotated position for predetermined period oftime (e.g., 2-5 seconds) to lock the representation of the camera datawith a particular magnification. Alternatively or in addition, in someembodiments, a user 115 is required to hold their wrist 382 in a rotatedposition to trigger the magnification of the camera data.

FIG. 3E illustrates further adjustments to the camera data while theuser 115 performs and maintains a hand gesture and performs a furthercamera-controlling gesture. In particular, while the user 115 performsthe pinch and hold gesture 251, the user performs a further gesture(e.g., rotating their wrist 382 by another predetermined degree) thatcauses the camera data captured by the imaging device 111 ofhead-wearable device to update. Specifically, the user 115 rotates theirwrist 382 in an opposite direction than the one shown in FIG. 3C (e.g.,clockwise), which causes the magnification of the camera data capturedby the imaging device 111 of head-wearable device to decrease (e.g.,zooms-out from 400% to 200%).

FIG. 3F illustrates a field of view 134 of the user 115 in FIG. 3E. Inparticular, in accordance with a determination the pinch and holdgesture 251 is performed and maintained, and a further cameracontrolling gesture is performed (e.g., the wrist 382 rotation in theopposite direction), the preview 260 presented via a display 130 of thehead-wearable device is updated to show a zoomed-out representation ofthe camera data. The changes to the preview 260 are presented to theuser while the pinch and hold gesture 251 is performed and maintainedand the wrist rotation is performed.

FIG. 3G illustrates the user 115 no longer performing and maintaining ahand gesture (e.g., releasing the hand gesture). As described above inFIGS. 2A-2D, in some embodiments, when the user 115 releases the handgesture, the camera control-gesture associated with the hand gesture iscompleted or a follow-on action is performed. In FIG. 3G, releasing thepinch and hold gesture 251 causes the head-wearable device 110 tocapture camera data with a magnification of 200%. In some embodiments, arepresentation of the captured image data is presented to the user 115via the display 130 of the head-wearable device 110.

FIG. 3H illustrates a field of view 134 of the user 115 in FIG. 3G.Specifically, the head-wearable device 110 presents, via display 130, acaptured image when the pinch and hold gesture 251 is released. Asdescribed above in reference to FIGS. 1D and 2D, the representation ofthe camera data (e.g., captured image 160) can be presented within auser interface 165 displayed at the display 130 of the head-wearabledevice 110. In some embodiments, the user interface 165 includes one ormore affordances for interacting with the captured image 160. Forexample, the one or more affordances can include, but are not limitedto, a save affordance 171, a share affordance 172, a messagingaffordance 173, an editing affordance 174, and a contacts affordance175. The save affordance 171 allows the user 115 to save the capturedimage 160 to memory, the share affordance 172 allows the user 115 todirectly share the captured image 160 to a website, application, socialmedia platforms, etc., the messaging affordance 173 allows the user todirectly send the captured image 160 via a messaging application, anediting affordance 174 allows the user to edit the captured image 160(e.g., apply filters, change the resolution, apply an additional zoom-inor zoom-out, tag objects or individuals, crop the captured image data160, and a number of other edits), and the contacts affordance 175allows the user to directly send the captured image 160 to a specifiedcontact.

In some embodiments, the user is able to perform other in-air handgestures to allow for quick sharing of captured images, without the needto display the one or more options. In another embodiment, both in-airhand gestures and presented affordances can be utilized to sharecaptured images 160. In one example, the user can contact a particularphalange portion of their middle finger to share with differentcontacts, e.g., if a distal phalange portion is contacted (e.g., using auser's thumb (phalange 140)), then the image shared with a firstcontact, if a middle phalange portion is contacted (e.g., using a user'sthumb (phalange 140)), then the image is shared with a second contactdistinct from the first contact, and if a proximal phalange portion iscontacted (e.g., using a user's thumb (phalange 140)), then the image isshared with a third contact that is distinct from the first and secondcontacts. In some embodiments, the in-air gesture that causes capturingof the image can be a multi-step in-air gesture such that a firstportion of the gesture (e.g., thumb (phalange 140) contacting indexfinger) can cause capturing of the image and a second portion of thegesture (e.g., a pressing force between the thumb (phalange 140) andindex finger (phalange 150) exceeding a force threshold) can causesharing of the image with a specific contact (e.g., different contactscan be associated with different force thresholds to allow for easysharing of images using such multi-step in-air gestures.

FIG. 3I illustrates the user 115 performing one or more gestures tonavigate and select one or more affordances presented within the userinterface 165. In particular, FIG. 3I shows the user 115 moving theirwrist 382 up or down to cycle through the one or more affordances. Forexample, as shown in FIG. 3J, in accordance with the user 115's wristmovement, the save affordance 171 is de-selected and the shareaffordance 172 is selected. The user 115 can select the affordancethrough the use of another hand gesture (e.g., contacting at least twophalanges, holding at least one phalange still, and/or other examplesdescribed herein.

FIG. 3K illustrates the user 115 performing one or more gestures tonavigate and select one or more contact affordances (e.g., shown in acontacts user interface 315) for sharing captured image data. The user115 can cause the head-wearable device 110 to display the contacts userinterface 315 via a punch and hold gesture 251. The one or more contactaffordances can include a first contact 325, a second contact 327, and athird contact 329 presented within the user interface 165. More thanthree contact affordances can be displayed within the user interface165. The user 115 can navigate through the different contact affordancesby moving their wrist 382 up or down or rotating their wrist 382 tocycle through the one or more affordances.

In FIG. 3L-1 , the first contact 325 is highlighted. The user 115 cannavigate through the different contact affordances by maintaining thehold gesture 251 and moving their wrist 382 up or down, or rotatingtheir wrist 382 to cycle through the one or more affordances. Forexample, in FIGS. 3L-2 and 3L-3 , while maintaining the pinch and holdgesture 251, the user 115 rotates their wrist 382 a predetermined amount(e.g., 20 degrees) to cycle from the first contact affordance 325 to thesecond contact affordance 327. In FIGS. 3L-4 and 3L-5 , whilemaintaining the pinch and hold gesture 251, the user 115 returns theirwrist 382 to the starting position which causes the contacts userinterface 315 to cycle back from the second contact affordance 327 tothe first contact affordance 325. In FIGS. 3L-6 and 3L-7 , the userselects the first contact affordance 325 by releasing the pinch and holdgesture 251. After the contact affordance is selected, the head-wearabledevice 110 can display a confirmation user interface 317 that allows theuser 115 to confirm sharing the captured image 160 or cancel (or goback).

FIGS. 3M and 3N illustrate a messaging interface 335 displayed at thedisplay 130 of the head-wearable device 110. The messaging interface 335can be displayed, via the head-wearable device 110, after the user 115has selected the first contact affordance 325 in FIG. 3L-7 . In someembodiments, a notification is provided to the user 115 to inform themthat the message was successfully sent to the selected contact (e.g.,“Image Sent” 337 or “Image sent to Mary S.”). The messaging interface335 includes a message thread shared between the user 115 and theselected contact affordance (e.g., the first contact 325). As shown inFIG. 3N, the captured image 160 is shared with the first contact 325 viathe message thread on interface 335. The captured image 160 can beshared using different applications or contact information. For example,the captured image 160 can be shred using one or more social mediaapplications, email, messaging applications, file sharing applications,etc.

FIGS. 3O-3Z illustrate selection of one or more capture modes using oneor more hand gestures performed by a user, in accordance with someembodiments. In particular, hand gestures that, when detected by awrist-wearable device 170, cause a head-wearable device 110 and/or thewrist-wearable device 170 to present one or more affordances forselecting one or more capture mores and/or an image data gallery asdiscussed in detail below. As described above in reference to FIGS.1A-1F, the wrist-wearable device 170 can detect and determine one ormore hand gestures based on sensor data obtained by one or more sensors1121 (FIG. 11 ).

In FIG. 3O, a pinch and hold gesture 251 associated with a cameracontrol-gesture for selecting one or more capture modes is detected. Thecamera control-gesture for selecting one or more capture modes, whendetected, causes the head-wearable device 110 to present, via a display130, one or more affordances for selecting a capture mode and/or viewingan image data gallery. For example, the head-wearable device 110 candisplay an image data capture affordance 383, a video data captureaffordance 384, and an album affordance 385 (FIG. 3R). As shown below inFIG. 3P, in addition to the one or more affordances for selection acapture mode and/or viewing an image data gallery, the head-wearabledevice 110 can display a representation of camera data captured by animaging device 111 of head-wearable device 110.

FIG. 3P illustrates a field of view 134 of the user 115 in FIG. 3O. Insome embodiments, in accordance with a determination the hand gesture isa camera control-gesture for selecting one or more capture modes, thehead-wearable device 110 presents, via the display 130, a preview 340,an image data capture affordance 383, a video data capture affordance384, an album affordance 385. Similar to FIGS. 2A-3N, the preview 340includes a representation of camera data captured within a field of view134 of the user 115. In some embodiments, the preview 340 includesinformation corresponding to the camera data captured by the imagingdevice 111. In some embodiments, the head-wearable device 110 alsopresents one or more indications that notify the user 115 that thehead-wearable device 110 is capturing imaging data (e.g., glasses icon231), capturing audio (e.g., mic icon 232), and/or the presenting audiodata (e.g., speaker icon 233).

FIG. 3Q illustrates the user 115 navigating between the differentaffordances for selecting a capture mode and/or viewing an image datagallery. In some embodiments, the user can navigate between thedifferent affordances for selecting a capture mode and/or viewing animage data gallery via an additional hand gesture, such as moving theirwrist 382 up and down, or left and right.

FIG. 3R illustrates a field of view 134 of the user 115 in FIG. 3C. Inparticular, in accordance with a determination the additional handgesture for navigating through the different affordances for selecting acapture mode and/or viewing an image data gallery is performed, thewrist-wearable device 170 and/or the head-wearable device 110 cyclethrough the different affordances. For example, as shown in FIG. 3R, theimage data capture affordance 383 is no longer highlighted and the videodata capture affordance 384 is highlighted. Selection of a particularcapture mode causes the head-wearable device 110 to capturecorresponding image data. For example, selection of the image datacapture affordance 383, causes the head-wearable device 110 to captureimage data as discussed above in reference to FIGS. 1A-3N.Alternatively, selection of the video data capture affordance 384,causes the head-wearable device 110 to capture video data as discussedbelow. Selection of the album affordance 385 causes the head-wearabledevice to display one or more captured images and/or videos.

In FIGS. 3S and 3T the user has selected the video data captureaffordance 384, via one or more hand gestures (e.g., a pinch gesture351). The wrist-wearable device 170, in response to determining that thevideo data capture affordance 384 is selected via one or more handgestures, causes the head-wearable device 110 to capture video data viaimaging device 111. The video data captured by the head-wearable device110 includes a portion of the user 115's field of view 134. Thehead-wearable device 110 continues to capture video data until the user115 provides another hand gesture associated with another camera-controlgesture for terminating the video capture. The head-wearable device 110can present, via display 130, the preview 340 of the captured video datato the user 115. While the head-wearable device 110 captures the videodata, the user 115 can perform additional hand gestures to adjust thecapture of the video data. For example, the user 115 can perform one ormore hand gestures for controlling the zoom of the video data capture asdiscussed above in reference to FIGS. 3A-3N.

FIGS. 3U-3V illustrates adjustments to the video data when one or morecamera-control gestures performed by the user 115 are detected. Inparticular, while the user 115 performs the pinch and hold gesture 251and rotates their wrist 382, the head-wearable device 110 is caused tozoom in on a portion of the user 115's field of view 134 (e.g., from100% to 400% as shown in preview 340) causes the camera data captured bythe imaging device 111 of head-wearable device to update.

FIGS. 3W-3X illustrates the user 115 performing an additional handgesture to end the video capture. In particular, the user 115 canperform another pinch gesture 351 that when detected by thewrist-wearable device 170 causes the head-wearable device to ceasecapturing video data. When the head-wearable device 110 ceases tocapture vide data, it displays a notification to the user that the videocapture session has ended (e.g., “Video Capture End). Additionally oralternatively, the head-wearable device 110 can also cease presentingthe glasses icon 231, the mic icon 232, and/or the speaker icon 233 suchthat the user 115 is aware that the head-wearable device 110 is notactively capturing audio and/or camera data. In some embodiments, theglasses icon 231, the mic icon 232, and/or the speaker icon 233 can bepresented with a strikethrough to further make clear to the user 115that data is not being captured.

FIG. 3Y illustrates the user 115 no longer performing a hand gesture(e.g., releasing the pinch gesture 351). As described above in FIGS.2A-3N, in some embodiments, when the user 115 releases the hand gesture,the camera control-gesture associated with the hand gesture is completedor a follow-on action is performed. In FIG. 3Y, releasing the pinchgesture 351 causes the head-wearable device 110 to present a userinterface for interacting with the captured video 350. For example, asshown in FIG. 3Z, the head-wearable device 110 presents, via display130, the captured vide data 350 and one or more affordances forinteracting with the captured video data 350, such as a save affordance171, a share affordance 172, a messaging affordance 173, an editingaffordance 174, and a contacts affordance 175. Additional information onthe one or more affordances for interacting with the captured video data350 (of capture image data 160) is provide above in reference to FIGS.1A-3N.

FIG. 4A-4J illustrate different surface-contact gestures performed by auser 115, in accordance with some embodiments. In particular,surface-contact gestures that are determined by a wrist-wearable device170 to be one or more camera-control gestures. As described above inreference to FIGS. 1A-3J, gestures are determined to be one or morecamera control-gestures based on sensor data obtained by one or moresensors 1121 (FIG. 11 ) of the wrist-wearable device 170.

In FIG. 4A, a user 115 is viewing a physical environment including oneor more objects while wearing the head-wearable device 110 and thewrist-wearable device 170. While wearing the wrist-wearable device 170,sensor data is monitored to sense a surface-contact gesture performed byuser 115. The data used to detect and determine a surface-contactgesture can also be data from an imaging device 111 that is able to seethe user 115's hand to use computer vision to detect gestures. One orall of the sensors can be utilized to provide data for detecting anddetermining a hand gesture. The sensed surface-contact gestures arefurther determined as a particular control gesture (e.g., acamera-control gesture) based, in part, on the sensor data. For example,when the user 115 moves a portion of his hand or intends to move aportion of his hand (e.g., phalanges 140, 150, etc.), the wrist-wearabledevice 170 detects position and/or orientation data as well asneuromuscular signals generated by the user 115's hand movement andcontact with a surface 410 (e.g., a table). The wrist-wearable device170, using the one or more processors 1150, determines the type of handmovements performed or intended to be performed by the user 115 as wella control command associated with the surface-contact gesture based onthe detected position and/or orientation data and neuromuscular signals.As described below, the wrist-wearable device 170 provides one or morecommands associated with the surface-contact gesture to another device,such as the head-wearable device 110, to perform a corresponding action.Alternatively, in some embodiments, the wrist-wearable device 170provides sensor data associated with the surface-contact gesture toanother device such that the other device can determine the type ofsurface-contact gesture performed by the user 115.

FIG. 4B illustrates the field of view 134 of the user 115 in FIG. 4A. Inparticular, the user 115's view before any surface-contact gesture isperformed and detected by the wrist-wearable device 170. The user 115can move around freely without having their field of view 134obstructed. For example, as the user 115 moves their head the field ofview 134 of the user 115 changes based on the user 115's movements. Asdescribed above, in some embodiments, the display 130 is coupled with alens or both lenses. In some embodiments, the display 130 is positionedover a portion of the lens without obstructing the user 115's view.Alternatively, in some embodiments, the display 130 makes up all or asubstantial portion of the lens or lenses.

In FIG. 4C, the user 115 performs and maintains a surface-contactgesture (e.g., pressing and holding phalange 430 on a surface). In someembodiments, maintaining a surface-contact gesture can include holding aphalange or phalanges pressed against a surface for a predeterminedperiod of time (e.g., two or more seconds). Alternatively oradditionally, in some embodiments, maintaining a surface-contact gesturecan include applying a predetermined threshold pressure between at leastone phalange and a surface (e.g., a table, wall, palm, leg, etc.) for apredetermined period of time (e.g., at least two seconds). Thesurface-contact gesture performed by the user 115 generates sensor data(e.g., EMG data, IMU data, etc.) that is used by the wrist-wearabledevice 170 to determine, using the one or more processors 1150 (FIG. 11), a camera control-gesture.

In FIG. 4C, a press and hold gesture 435 is performed by the user 115.In some embodiments, similar to the camera-control gesture describedabove in reference to FIGS. 2A-2D, the press and hold gesture 435 isassociated with a camera control-gesture for displaying a preview of arepresentation of camera data captured by an imaging device 111. Thecamera control-gesture for displaying a preview of a representation ofcamera data captured by an imaging device 111, when detected by thewrist-wearable device 170, causes the head-wearable device 110 topresent, via a display 130, a representation of the camera data capturedby an imaging device 111.

FIG. 4D illustrates a field of view 134 of the user 115 in FIG. 4C. Insome embodiments, in accordance with a determination the hand gesture isa camera control-gesture for displaying a preview of a representation ofcamera data captured by an imaging device 111 (e.g., the press and holdsurface-contact gesture 435), the head-wearable device 110 presents apreview 260 to the user 115. The preview 260 includes a representationof camera data captured within a field of view 134 of the user 115. Insome embodiments, the preview 260 includes a subset of the field of view134 or of the entire field of view 134 of the user 115. The preview 260can be a live feed (e.g., simultaneous capture and presentation) of whatthe camera is viewing while the hand gesture is maintained.Alternatively, the preview 260 can be a recording of a field of view 134captured by the user 115. In some embodiments, a plurality of previewsfrom one or more cameras can be displayed on the head-wearable device110. In some embodiments, the preview 260 is presented to the user 115via a user interface 165 as described above in reference to FIGS. 1A-3J.

FIG. 4E illustrates the user 115 performing a surface-contactcamera-control gesture intended to switch to a second coupled imagingdevice, that is detected by the wrist-wearable device 170. In someembodiments, the camera-control gesture is associated with instructionsto cause an imaging device 111 to switch between different cameras(e.g., a second imaging device 420, such as a security camera; asecondary camera embedded in a head-wearable device; a wrist-wearablecamera, etc.) coupled to the head-wearable device, wrist-wearable device170, or other intermediary device.

In some embodiments, the camera-control gesture intended to switchbetween devices is based on a characteristic of the hand gesture. Insome embodiments, in accordance with a determination that one or morecharacteristics of the hand gesture are within a first threshold, thewrist-wearable device 170 causes a first camera to capture the cameradata for presentation at the display of the head-wearable device that isworn by the user; and in accordance with a determination that the one ormore characteristics of the hand gesture are within a second threshold,the wrist-wearable device 170 causing a second camera to capture thecamera data for presentation at the display of the head-wearable devicethat is worn by the user. In some embodiments, the method 700 includesdetermining, based at least on the received data from the one or moresensors, one or more characteristics of the hand gesture and performinga different action (e.g., camera-control gesture) based on thecharacteristics of the performed hand gestures. In some embodiments, theone or more characteristics of the in-air hand gesture include one ormore of a force, a duration, a number of contacts, and a pattern (e.g.,sequence of taps).

In some embodiments, in accordance with a determination, by thewrist-wearable device 170 that the sensor data indicates that thesurface-contact gesture is a camera-control gesture for switching to analternate camera source to display different camera data, thewrist-wearable device 170 provides one or more commands or instructionsassociated with the camera-control gesture to another device includingor communicatively coupled with an imaging device 111, such as thehead-wearable device 110. The commands or instructions associated withthe camera-control gesture cause the second imaging device 420 of theother device to capture camera data. For example, when the user 115presses phalange 430 onto a surface with additional force (e.g., pressharder), the surface-contact gesture is detected by the wrist-wearabledevice 170, the head-wearable device 110 is caused to display cameradata from the imaging device 420 (as shown by preview 260). Morespecifically, the camera-control gesture demonstrated in FIG. 4E is theuser 115's pointer finger (e.g., phalange 430) pressing harder on asurface 410 which ultimately results in the preview of the camera datafrom the second imaging device 420 (e.g., the preview 260 now shows theuser 115 which is in the field of view 421 of a second imaging device420). The user can switch to any number of imaging devices coupled tothe head-word device, wrist-wearable device 170, smartphone or any otherintermediary device. The head-wearable device can also display multiplerepresentations of multiple different camera's (e.g., could have one ormore security cameras and see each of them displayed in the display ofthe head-wearable device, or rotate through them to see one imagerepresentation at a time.) The user 115 can also combine a combinationof in-air hand gestures and surface-contact gestures to control thecameras.

The above examples are non-limiting; the user 115 could perform a pressand hold gesture 435 with other phalanges and is not only limited to thepointer finger (phalange 150) (e.g., could also be a thumb (phalange140), middle finger, ring finger, pinky, two other fingers, etc.).Similarly, the different characteristics of a hand gesture (e.g., force,duration, number of contacts, etc.) can apply to surface-contactgestures, in-air contact gestures, and/or any other gestures detectableby the wrist-wearable device 170.

FIG. 4F illustrates the field of view 421 of the user 115 in FIG. 4E.More specifically, FIG. 4F shows the user 115's view after thecamera-control gesture (e.g., the press and hold surface-contact gesture435) is detected by the wrist-wearable device 170. The wrist-wearabledevice 170, in response to determining that the surface-contact gestureis a camera-control gesture for switching to an alternate camera, causesthe head-wearable device 110 to display the camera data from the secondimaging device 420. In some embodiments, the camera-control gesture forswitching to a separate camera also causes the head-wearable device 110to present a representation of the camera data (e.g., preview 260). Insome embodiments, the representation of the camera data from 420 isdisplayed over a portion of the user 115's field of view 134 asdescribed above in reference to FIGS. 1A-1F.

FIG. 4G illustrates adjustments to the camera data from an imagingdevice 420 while the user 115 performs and maintains a surface-contactgesture. In particular, while the user 115 performs the press and holdgesture 435 with a greater among of force (e.g., the cameracontrol-gesture for displaying a preview of a representation of cameradata from an imaging device 420), the camera data captured by the secondimaging device 420 of a second camera updates in accordance with changesto the user 115's movements. For example, while the user 115 performsand maintains the press and hold gesture 435 with greater force, thesecond imaging device 420 captured camera data reflecting changes in theuser 115's head movement (e.g., moving side-to-side up and down, etc.).Alternatively or in addition, in some embodiments, while the user 115performs and maintains the press and hold gesture 435, movement of theuser 115's wrist can be interpreted to cause changes in the field ofview 421 of the camera (e.g., panning the camera side-to-side, up anddown, etc.).

FIG. 4H illustrates a field of view 421 of the user 115 in FIG. 4G. Inparticular, in accordance with a determination the press and holdgesture 435 is performed and maintained, the preview 260 presented via adisplay 130 of the head-wearable device is updated to show arepresentation of the camera data reflective of the second imagingdevice's 420 field of view 421 as they move their head. For example, asthe user 115 moves their head down, the preview 260 is updated to showthe second imaging device's 420 field of view 421 (e.g., the bottom ofthe user 115 as opposed to the top). The changes to the preview 260 arepresented to the user while the press and hold gesture 435 is performedand maintained. In this way, the user 115 has a visual representation ofthe camera data that is being captured or about to be captured by theimaging device 420 which is displayed the head-wearable device 110.

FIG. 4I illustrates the user 115 no longer performing and maintaining asurface-contact gesture (e.g., releasing the surface-contact gesture).Releasing a surface-contact gesture can include moving a phalange from asurface that was held still for a predetermined period of time;separating at least two phalanges that were contacting a surface suchthat they are no longer in contact with the surface; moving a phalangeor phalanges contacting a surface such that the phalange or phalangesare no longer contacting the surface; and/or removing (e.g., no longerapplying) a predetermined threshold pressure between at least onephalange and a surface. The wrist-wearable device 170 determines thatthe user 115 releases the surface-contact gesture (e.g., press and holdgesture) based on sensor data obtained by the one or more sensors 1121.In some embodiments, when the user 115 releases the hand gesture, thecamera control-gesture associated with the surface-contact gesture iscompleted or a follow-on action is performed. For example, releasing thepress and hold gesture can cause the head-wearable device to capturecamera data, via an imaging device of the head-wearable device, of theuser 115's field of view when the press and hold gesture 251 isreleased. In some embodiments, a representation of the captured imagedata is presented to the user 115 via the display 130 of thehead-wearable device 110.

FIG. 4J illustrates a field of view 134 of the user 115 in FIG. 4I.Specifically, the head-wearable device 110 presents, via display 130, afield of view of the user 115 when the press and hold gesture isreleased. As described above, in some embodiments, when the press andhold gesture is released, the imaging device 111 of the head-wearabledevice captures camera data of the user 115's field of view. Further, insome embodiments, when the press and hold gesture is released, arepresentation of the captured camera data is presented to the user 115.

Although the above-examples reference surface-contact gestures,difference in-air gestures and/or other gestures that do not contact awrist-wearable device 170 are also contemplated for performing thedifferent operations described in reference to FIGS. 4A-4J.

FIGS. 5A-5C illustrate coordination of a wrist-wearable device 170 and ahead-wearable device 510 to cause presentation of camera data in an ARenvironment, in accordance with some embodiments. In particular, usinggestures detected at a wrist-wearable device 170 to cause presentationof a representation of virtual environment presented by a head-wearabledevice 510. In some embodiments, the wrist-wearable device 170 iscommunicatively coupled with the head-wearable device 510 (e.g., by wayof a Bluetooth connection between the two devices, and/or the twodevices can also both be connected to an intermediary device such as asmartphone that provides instructions and data to and between the twodevices).

As described above in reference to FIGS. 1A-4J, while the user 115 wearsthe wrist-wearable device 170, sensor data is monitored to sense a handgesture performed by user 115. The sensor data is used detect anddetermine a hand gesture performed by the user 115. In FIG. 5A, thehead-wearable device 510 presents, via its display 130, a virtualenvironment seen by the user 115 (e.g., the user 115's character 521,avatar 517, health bar 527, and endurance bar 529). The wrist-wearabledevice 170 and the head-wearable device 510 are configured to performone or more operations described above in reference to FIGS. 1A-4J.

In FIG. 5B, the user 115 performs a camera-control gesture that isdetected by the wrist-wearable device 170. In accordance with adetermination, by the wrist-wearable device 170 that the sensor dataindicates that the hand gesture is a camera-control gesture forcapturing camera data, the wrist-wearable device 170 causes thehead-wearable device 510 to capture a captured image 565 of the virtualenvironment as seen by the user 115's character 521. As described abovein reference to FIGS. 1A-1F, the captured image 160 can be presented tothe user via the display of the head-wearable device 510. The capturedimage 565 can be included in a user interface 165 that includes one ormore affordances for sharing, editing, and saving the captured image. Insome embodiments, the head-wearable device 510 presents to the user 115one or more indications that notify the user 115 that the head-wearabledevice 510 is capturing imaging data (e.g., glasses icon 231), capturingaudio (e.g., mic icon 232), and/or the presenting audio data (e.g.,speaker icon 233).

FIG. 5C illustrates an example gesture performed by the user 115 that isnot recognized as a camera-control gesture. In some embodiment, thewrist-wearable device receives, from one or more sensors 1121 (FIG. 11), additional data generated based on performance of a hand gesture bythe user, and determines, based on the additional data, that the handgesture is not a camera-control gesture. The wrist-wearable device 170,in accordance with the determination that the hand gesture is not acamera-control gesture, forgoes causing presentation of camera data viathe display of the head-wearable device 510 and activates another(non-camera related) operation of the head-wearable device 510. Forexample, in FIG. 5C, the hand gesture 535 causes a microphone of thehead-wearable device to activate (as indicated by the mic icon 232). Inaddition or alternatively, in some embodiments, the non-camera-controlgesture activates the microphone to allowing the user to provide a voicecommand to a digital assistant. In some embodiments, the non-camerarelated operation remains active as long as the additional dataindicates that the hand gesture 535 is maintained. For example, as longas the user 115 maintains the hand gesture 535, the microphone of thehead-wearable device 510 remains active. When the additional dataindicates that the hand gesture 535 is no longer maintained (e.g.,released), the non-camera related operation is terminated ordeactivated. For example, when the user 115 no longer maintains the handgesture 535, the microphone of the head-wearable device 510 isdeactivated. The above example is non-limiting; additional non-camerarelated operations can be performed, such as opening and closing one ormore applications, initiating an audio call, and/or other device andapplication specific controls.

FIGS. 6A-6B illustrates a flow diagram of a method for detecting anin-air hand gesture and determining if the in-air hand gesture is acamera-control gesture, according to some embodiments. Operations (e.g.,steps) of the method 600 can be performed by one or more processors(e.g., central processing unit and/or MCU; processors 1150, FIG. 11 ) ofa wrist-wearable device 170. In some embodiments, the wrist-wearabledevice 170 is coupled with one or more sensors (e.g., various sensorsdiscussed in reference to FIG. 11 , such as a heart rate sensor, IMU, aneuromuscular-signal sensors (e.g., EMG sensors or other sensors), SpO2sensor, altimeter, thermal sensor or thermal couple, ambient lightsensor, ambient noise sensor), a display, a speaker, an image sensor(e.g., imaging device; FIGS. 2A-21 ), and a microphone to perform theone or more operations. At least some of the operations shown in FIGS.6A and 6B correspond to instructions stored in a computer memory orcomputer-readable storage medium (e.g., storage, ram, and/or memory,FIG. 11 ). Operations of the method 600 can be performed by thewrist-wearable device 170 alone or in conjunction with one or moreprocessors and/or hardware components of another device communicativelycoupled to the wrist-wearable device 170 (e.g., a head-wearable device110, a smartphone, a laptop, a tablet, etc.) and/or instructions storedin memory or computer-readable medium of the other devicecommunicatively coupled to the wrist-wearable device 170.

The method 600 includes receiving (602) data generated based onperformance of a hand gesture by a user. For example, in someembodiments the hand gesture by the user 115 can be a pinch action wheretwo of the user's phalanges including but not limited to the user'sthumb (phalange 140) and pointer finger (phalange 150) squeeze togetherin a pinching action. Another example of a hand gesture is where theuser presses their thumb (phalange 140) and pointer finger (phalange150) together maintaining the pinch motion for a period of time beforereleasing. For these gestures, the user 115 can use any phalange or evenother parts of the body. For example, in some embodiments the user 115can use their pointer finger (phalange 150) to press on a surface 410such as a table for a short or longer period of time to make the handgesture. Example gestures are shown and described above in reference toFIGS. 1A-5C

The method 600 further determining (604) whether the hand gestureperformed by the user 115 is a camera-control gesture. A camera-controlgesture can be assigned to a hand gesture by the user or automaticallyassigned by a user device. An example of a camera-control gesture isdescribed in FIG. 2A, where the user 115 performs an in-air hand gestureby performing a pinch and hold gesture 251 which controls the camera byproviding the user with a picture preview displayed on the head-wearabledevice 110. However, if the user is not performing a camera-controllinghand gesture (“No” at operation 604), then the method 600 includesperforming (606) a non-camera operation. Examples of a non-cameraoperation are turning on the microphone on a device (e.g., thewrist-wearable device 170, head-wearable device, smartphone, etc.) toallow the user to provide an audio input as shown in FIGS. 5A-5C.

In accordance with a determination that the user is performing acamera-control gesture (“Yes” at operation 604), the method furtherincludes determining (608) whether the hand gesture is maintained. Asdescribed above in reference to FIGS. 2A-3J, a hand gesture ismaintained if it is held still for a predetermined period of time, aforce is applied for a predetermined period of time, at least twophalanges or at least a phalange and a surface are in contact for aperiod of time. Alternatively, as shown in FIGS. 1A-1F, a hand gestureis not maintained when it is released immediately (e.g., less than asecond) after being performed.

In accordance with a determination (608) that the hand gesture is notmaintained (“No” at operation 608), the method 600 includes causing(610) presentation of a representation of camera data from a camera to auser via a display that is in communication with one or both of thewrist-wearable device 170 and the head-wearable device. In someembodiments, the representation of camera data is an image. In someembodiments, the method 600 further includes causing (612) thepresentation one or more options for sharing a respective representationof the camera data to the user. Additional options can include savingthe image, sending the image to a contact via a messaging application,editing the image in an application or directly through the presentinginterface, or sharing it with a contact directly. Additional examples ofthe options (also referred to as “affordances”) are provided above inreference to FIGS. 3A-3J. In some embodiments, repulsive to the userselection, the method 600 includes sharing (614) the respectiverepresentation of the camera data. For example, in some embodiments, ifthe user 115 selects sharing the image via a messaging application(e.g., selection of messaging affordance 173), then a messagingapplication will open on the heads-up display, wrist-wearable device170, smart phone, or other intermediary device, which allows the user115 to select a contact to send the image to. After the respectiverepresentation of the camera data is shared, the method 600 returns tooperation 602 and waits to receive additional data generated by theperformance of a hand gesture by the user.

Returning to operation 608, in accordance with a determination that thehand gesture is maintained (“Yes” at operation 608), the method 600includes continuously causing (616) presentation of a representation ofcamera data of a field of view of the camera to the user via the displayof the head-wearable device (while the hand gesture is maintained). Insome embodiments, as long the camera controlling hand gesture ismaintained (e.g., holding the pinch motion for some period of time), thepresentation of camera data acts as a live feed and will displaywhatever the camera that is coupled to the wrist-wearable device 170and/or head-wearable device is seeing. For example, if the active camerais integrated with the head-wearable device, then as the user movesaround or as the user moves their head up or down the picture previewwill continue to update immediately to where the camera is pointed ormoved. For example, FIG. 2D illustrates the new preview 260 as a resultof the user's head movement. For example, as shown in FIG. 2D the user115 moved their head down so the preview 260 now shows the bottom of theobject as opposed to the top. In other embodiments, the user could belooking in an alternate location and have moved their head or wrist inany desired direction by the user 115 and the picture preview wouldupdate immediately.

In some embodiments, the method 600 further includes determining (618)whether the hand gesture is released. In accordance with a determinationthat the hand gesture is released (“Yes” at operation 618), the methodincludes causing (624) the presentation of a representation of cameradata of a field of view of the camera to the user via the display of thehead-wearable device while the hand gesture was maintained. In otherwords, an image of the representation of the camera data was capture atthe moment that the user releases the hand gesture. As illustrated inFIG. 2F, once the user releases the gesture, an image is captured (e.g.,captured image 160) and presented to the user via a display 130 of ahead-wearable device 110. The method 600 further proceeds to perform atleast operations 612 and 614.

Returning to operation 618, in accordance with a determination that thehand gesture is not released (“No” at operation 618), the method 600includes determining (620) whether an additional hand gesture forcontrolling the camera is detected. In accordance with a determinationthat an additional hand gesture for controlling the camera is detected(“Yes” at operation 620), the method 600 includes causing (622) one ormore adjustments to the representation of the camera data of a field ofview of the camera. For example, as described above in reference toFIGS. 3A-3J, while the user 115 maintains a pinch gesture (e.g., withtheir pointer finger (phalange 150) and thumb (phalange 140)), if theuser rotates their wrist 382, an additional camera-control gesture isdetected, which causes the representation of the camera data to zoom-inor zoom-out. After one or more adjustments to the representation of thecamera data of a field of view of the camera have been performed, themethod 600 returns to operation 618. Returning to operation 620, inaccordance with a determination that an additional hand gesture forcontrolling the camera is not detected (“No” at operation 620), themethod 600 returns to operation 618.

FIGS. 7A and 7B illustrates a detailed flow diagram of a method of usinga gesture detected based on data from a wrist-wearable device 170 tocause presentation of camera data at a head-wearable device, accordingto some embodiments. Similar to method 600 of FIGS. 6A and 6B,operations of the method 700 can be performed by one or more processorsof a wrist-wearable device 170. At least some of the operations shown inFIGS. 7A and 7B correspond to instructions stored in a computer memoryor computer-readable storage medium. Operations of the method 700 can beperformed by the wrist-wearable device 170 alone or in conjunction withone or more processors and/or hardware components of another device(e.g., a head-wearable device 110 and/or an intermediary devicedescribed below in reference to FIG. 11 ) communicatively coupled to thewrist-wearable device 170 and/or instructions stored in memory orcomputer-readable medium of the other device communicatively coupled tothe wrist-wearable device 170.

Method 700 includes receiving (710) data generated based on performanceof an in-air hand gesture by a user that is wearing a wrist-wearabledevice. The wrist-wearable device can include one or more sensors fordetecting the data generated based on performance of an in-air handgesture. The data can be neuromuscular signal and/or IMU data. In someembodiments, the data can also be data from a camera that is able to seethe user's hand to use computer vision to detect gestures. One or all ofthe sensors of the wrist-wearable device 170 can be utilized to receivedata on the performance of an in-air hand gesture.

In some embodiments, “in-air” means, in some embodiments, that theuser's hand does not contact a portion of the device. In other words thegesture is performed in open air in 3D space and without contacting thewrist-wearable device 170. The in-air gestures are performed without aglove, without contacting a glove surface, or without the wrist-wearabledevice 170 being coupled to a glove. In some embodiments, the in-airhand gesture can include a single contact between a distal phalange(distal, proximal, middle) phalange portion of the user's thumb(phalange 140) and a distal phalange portion of the user's index finger,which causes selection of a first camera (e.g., a camera of thehead-wearable device), or the in-air hand gesture includes two or morecontacts between a distal phalange portion of the user's thumb (phalange140) and a distal phalange portion of the user's index finger, which cancause selection of a second camera distinct from the first camera (e.g.,a security camera located near (e.g., within a distance that allows forshort-range communication) the wrist-wearable device 170). Such in-airfinger-contact gestures are non-limiting examples of the in-air gesturescontemplated herein, other examples include gestures in which twofingers (and phalange portions thereof) are moved to make contact with athumb (phalange 140) (and a phalange portion thereof), multi-stepgestures in which a first digit can make contact with a thumb and remainin contact and then another digit can make contact with the first digitwhile the first digit remains in contact with the thumb. Rotationalin-air hand gestures are also contemplated, in which a user can rotatetheir wrist in various directions.

Surface-contact gestures (contacts at surfaces other than thewrist-wearable device 170) more generally are also contemplated in whichneuromuscular signals (or other types of data from other sensors, suchas proximity sensors, time-of-flight sensors, sensors of an IMU, etc.)can be used to detect a contact (or an intention to contact) a surface(e.g., a single or double finger tap on a table, on a user's leg, acouch, etc.). Thus, while the primary example herein is an in-airgesture, the disclosure is not limited to those in-air gestures, asother gestures that do not contact a wrist-wearable device 170 are alsocontemplated, including the surface-contact gestures just described. Insome embodiments, the one or more sensors of the wrist-wearable device170 detect data generated by one handed gestures performed by the user(e.g., detected from the hand wearing the wrist-wearable device 170).Examples of the different gestures are described above in reference toFIGS. 1A-5B.

The method 700 includes, in accordance with a determination (720) thatthe data indicates that the in-air hand gesture is a camera-controlgesture, causing (722) presentation to the user, via a display of ahead-wearable device that is worn by the user and that is incommunication with the wrist-wearable device 170, of a representation ofcamera data from a camera that is in communication with one or both ofthe wrist-wearable device 170 and the head-wearable device 110. Forexample, the wrist-wearable device 170 can send an instruction to thehead-wearable device to cause the display of the head-wearable device tobegin presenting the representation of the camera data. Alternatively orin addition, an intermediary device (that receives the data generatedbased on performance of an in-air hand from the wrist-wearable device)can send the instruction, or the head-wearable device can receive thedata generated based on performance of an in-air hand gesture, processthe data generated based on performance of the in-air hand gesture, andthen make the determination that a camera-control gesture performed tothen cause its display to begin presenting the representation of thecamera data. In some embodiments, the display of the head-wearabledevice is a heads-up display integrated with one or more lenses of thehead-wearable device. In some embodiments, the head-wearable device iscaused to present the representation of camera data directly andimmediately in response to the in-air gesture, which means that no otherinput is received from the user to trigger the presentation of therepresentation of camera data. Examples of the representation of thecamera data displayed by a head-wearable device are provided above inreference to FIGS. 1A-5C.

Non-limiting examples of the camera-control gestures include a doublepinch gesture performed when at least two phalanges of a user's handmake contact with one another, at least two phalanges of a user's handcontacting one another in succession, at least two phalanges of a user'shand making contact with one another and remaining in contact, wristrotations, wrist movements, and other hand movements.

In some embodiments, the camera-control gesture causes capturing of animage using the camera, and the causing presentation to the userincludes causing presentation, via the display of the head-wearabledevice, of the image captured by the camera. In some embodiments, theimage is one of a plurality of images, each respective image caused tobe captured in accordance with the determination that the data indicatesthat the in-air hand gesture is the camera-control gesture, and thecausing presentation to the user includes causing presentation ofrespective representations of each respective image of the plurality ofimages. The presentation of the image can occur at a display of thehead-wearable device or at a display of the wrist-wearable device 170.The device selected to present the image can be based on whether or notthe user is determined to be looking at the wrist-wearable device 170 ornot (e.g., if the user is determined to be looking at the wrist-wearabledevice 170, then the presentation can occur at the display of thatdevice; if not, then the display of the head-wearable device can beused).

In some embodiments, after the image is captured, the method 700includes causing presentation of one or more options for sharing theimage; and responsive to a user selection of a first option of the oneor more options for sharing the image, causing the image to be sent toanother electronic device. In some embodiments, the method 700 furtherincludes responsive to a user selection of a second option, distinctfrom the first option, of the one or more options for sharing the imagewith a specific contact, causing the image to be send to an electronicdevice associated with the specific contact. In some embodiments, amessaging application can be used to facilitate sharing of the image,which can include sending the image, using the messaging application,directly to a particular contact. The application can be an applicationthat is executing on one or both of the wrist-wearable device 170 andthe head-wearable device.

Presentation of options for sharing the image is one example but is anon-limiting example. In some embodiments, the user is able to performother in-air hand gestures to allow for quick sharing of captured imageswithout the need to display the one or more options (in otherembodiments, both in-air hand gestures and presented options can beutilized to share captured images). As one specific example, the usercan contact a particular phalange portion of their middle finger toshare with different contacts, e.g., if a distal phalange portion iscontacted (e.g., using a user's thumb), then the image is shared with afirst contact, if a middle phalange portion is contacted (e.g., using auser's thumb), then the image is shared with a second contact distinctfrom the first contact, and if a proximal phalange portion is contacted(e.g., using a user's thumb), then the image is shared with a thirdcontact that is distinct from the first and second contacts. In someembodiments, the in-air gesture that causes capturing of the image canbe a multi-step in-air gesture such that a first portion of the gesture(e.g., thumb contacting index finger) can cause capturing of the imageand a second portion of the gesture (e.g., a pressing force between thethumb and index finger exceeding a force threshold) can cause sharing ofthe image with a specific contact (e.g., different contacts can beassociated with different force thresholds to allow for easy sharing ofimages using such multi-step in-air gestures). Different examples ofoptions for sharing and editing an image are provided above in referenceto FIG. 3A-3J.

In some embodiments, the method 700 further includes selecting thecamera from among one or more available cameras that are communicativelycoupled to the wrist-wearable device 170 and/or the head-wearable device110 based on a characteristic of the in-air hand gesture. In someembodiments, in accordance with a determination that one or morecharacteristics of the in-air hand gesture are within a first threshold,the method 700 includes causing a first camera to capture the cameradata for presentation at the display of the head-wearable device that isworn by the user; and in accordance with a determination that the one ormore characteristics of the in-air hand gesture are within a secondthreshold, the method further includes causing a second camera tocapture the camera data for presentation at the display of thehead-wearable device that is worn by the user. In some embodiments, themethod 700 includes determining, based at least on the received datafrom the one or more sensors, one or more characteristics of the in-airhand gesture and performing a different action (e.g., camera-controlgesture) based on the characteristics of the performed hand gestures. Insome embodiments, the one or more characteristics of the in-air handgesture include one or more of a force, a duration, a number ofcontacts, and a pattern (e.g., sequence of taps, morse code, etc.).

In some embodiments, the user can control the cameras cameraconcurrently or separately. In some embodiment, the first camera is inproximity to the user and the second camera is remote to the user. Insome embodiments, proximity means within 3-5 feet of the user, andremote means at least more than 5 feet away from the user.Alternatively, in some embodiments, the first and second camera areremote from the user, or in proximity to the user. Control of devicesother than imaging devices is also contemplated. In some embodiments,different in-air hand gestures detected by the wrist-wearable device 170can cause the performance of different operations. For example, in-airhand gestures detected by the wrist-wearable device 170 can be used tocontrol a communicatively coupled tablet or computer. In someembodiments, in-air hand gestures detected by the wrist-wearable device170 can be used to open applications, using the wrist-wearable device170 as a controller for the head-wearable device, and number of otheroperations.

In some embodiments, the data is generated using, in part, one or moresensors of the wrist-wearable device, and while data generated by theone or more sensors of the wrist-wearable device 170 indicates that thein-air hand gesture is maintained, continuing (730) to cause thepresentation of the representation of the camera data via the display ofthe head-wearable device. Maintaining the in-air hand gesture means, insome embodiment, that phalange portions of the user's fingers remainingin contact with one another, which can be detected by analyzing signalinformation from neuromuscular-signal sensors of the wrist-wearabledevice 170. The maintaining can also, or alternatively, include acertain amount of applied pressure being maintained in conjunction withthe in-air hand gesture (a pressing force applied by one or both of thephalange portions against each other), which can force also be detectedby analyzing signal information from neuromuscular-signal sensors of thewrist-wearable device 170. In some embodiments, the continuing to causethe presentation of the representation of the camera data via thedisplay of the head-wearable device includes causing presentation of arepresentation of a field of view of the camera to the user via thedisplay of the head-wearable device. In some embodiments, while thein-air hand gesture is maintained, movement of the user's head ormovement of the user's wrist can be interpreted to causes changes in thefield of the view of the camera (e.g., side-to-side movement of the heador side-to-side movements of the wrist can cause the camera's field ofview to change, which then is immediately updated on the display of thehead-wearable device).

In some embodiments, the method 700 further includes, in accordance witha determination that the in-air hand gesture is no longer maintained,causing (732) the presentation of the representation of the camera datavia the display of the head-wearable device to cease. For example, whenthe in-air hand gesture is released such that the phalange portions areno longer in contact with one another or such that a threshold amount offorce is no longer applied by the phalange portions against one another,the method includes ceasing to cause presentation of the camera data. Insome embodiments, the wrist-wearable device 170 can send an instructionto the head-wearable device to cause the display of the head-wearabledevice to cease presenting the representation of the camera data.Alternatively or in addition, an intermediary device can send theinstruction, or the head-wearable device can process the data generatedbased on performance of the in-air hand gesture to determine that thein-air hand gesture is no longer maintained, and then cease presentingthe representation of the camera data. In some embodiments, the method700 includes in accordance with a determination (734) that datagenerated by the one or more sensors of the wrist-wearable device 170indicates that the in-air hand gesture is no longer maintained causing(735) a capture of an image within the field of view using the camera,and cause presentation of a representation of the image via the displayof the head-wearable device. Different examples for maintaining andreleasing a gesture are provided above in reference to FIGS. 2A-4J.

In some embodiments, the method 700 includes, responsive to adetermination that an additional in-air hand gesture is performed whilethe in-air hand gesture is maintained, causing (740) a zoomed-inrepresentation of the camera data to be presented via the display of thehead-wearable device. In some embodiments, in accordance with adetermination that the in-air hand gesture is no longer maintained whilethe zoomed-in representation of the camera data is being presented viathe display of the head-wearable device, causing (742) the camera tocapture the zoomed-in representation of the camera data. In someembodiments, causing (744) presentation to the user, via the display ofthe head-wearable device that is worn by the user, of an image of thezoomed-in representation of the camera data

In some embodiments, the method 700 includes, responsive to adetermination that a further in-air hand gesture is performed while thein-air hand gesture is maintained, causing (760) a zoomed-outrepresentation of the camera data to be presented via the display of thehead-wearable device. In some embodiments, the method 700 includes inaccordance with a determination that the further in-air hand gesture isno longer maintained while the zoomed-out representation of the cameradata is being presented via the display of the head-wearable device,causing (762) the camera to capture the zoomed-out representation of thecamera data. In some embodiments, causing (764) presentation to theuser, via the display of the head-wearable device that is worn by theuser, of an image of the zoomed-out representation of the camera data.

Examples of the zoomed-in and zoomed-out captures are provided above inreference to FIG. 3A-3J. While some embodiments can require the firstin-air hand gesture to be maintained to activate the zoom-in andzoom-out in-air hand gestures, other embodiments can instead (oradditionally) allow for activating the camera control/presentationfeatures, then allow for a release of the first in-air hand gesture,followed by then detecting the second in-air hand gesture to allowingfor controlling zoom features of the camera. In such other embodiments,a second detection of the first in-air hand gesture (e.g., at a point intime after the first detection of the in-air hand gesture thatoriginally triggered activation of the camera control/presentationfeatures) can cause the camera control/presentation features to thencease.

In some embodiments, the in-air hand gesture is a maintained contactbetween at least two phalanges of a user's hand, the additional in-airhand gesture is an in-air rotational movement of the user's wrist in afirst direction, and the further in-air hand gesture is an in-airrotational movement of the user's wrist in a second direction distinctfrom the first direction.

In some embodiments, the method 700 further includes receivingadditional data generated based on performance of yet another additionalin-air hand gesture by the user while the user is wearing thewrist-wearable device. The additional data generated based onperformance of yet another additional in-air hand gesture can bedetected by one or more sensors of the wrist-wearable device 170. Themethod also includes, in accordance with a determination that theadditional data indicates that the other additional in-air hand gestureis not a camera-control gesture, forgo causing presentation of cameradata via the display of the head-wearable device and cause activation ofa microphone for allowing the user to provide a voice command to adigital assistant. The microphone is in communication with one or bothof the wrist-wearable device and the head-wearable device. In otherwords, certain in-air gestures can command camera controls, while otherin-air hand gestures can command other actions at the head-wearabledevice. In some embodiments, the microphone of the head-wearable deviceremains active as long as the other additional in-air hand gesture ismaintained. In some embodiments, the method 700 includes, upondetermining that further data indicates that the other additional in-airhand gesture is no longer maintained, causing deactivation of themicrophone that is in communication with one or both of thewrist-wearable device 170 and the head-wearable device. In other words,the method includes deactivating the microphone upon receiving dataindicating that the in-air hand gesture was released.

In some embodiments, the captured camera data and images include atleast 10 megapixels (MP) or at least 12 MP. In some embodiments, thecamera data and images are captured in full high definition (e.g.,1080p). In some embodiments, the camera data and images are captured in1440p or higher. The resolution of the captured camera data and imagesallow the user to zoom-in and zoom-out as desired. In some embodiments,the camera data and images are scaled for the respective display thatpresents the camera data and images.

Although the above examples show the head-wearable device 110 displayingthe captured camera data and/or one or more user interface, in someembodiments, the wrist-wearable device 170 can also display the cameradata and/or one or more user interfaces.

FIGS. 8A-8H illustrate a head-wearable device including a vestigialmuscle training mode for training a user to use their vestigial earmuscles to perform vestigial ear muscle gestures that are detectable bythe head-wearable device. In FIG. 8A, a user 115 wearing a head-wearabledevice 110 provides a request to enable vestigial ear muscle detectionvia the head-wearable device 110. Responsive to the user 115's request,a vestigial muscle training mode of the head-wearable device 110 isinitiated to train the user 115 to use their vestigial ear muscles toperform vestigial ear muscle gestures. In some embodiments, the requestto initiate the vestigial muscle training mode is a verbal command, ahand gesture, and/or a touch input detected by the head-wearable device110, a wrist-wearable device 170, and/or an intermediary device. Forexample, the user 115, can navigate via one or more user interfacesand/or user interface elements presented by the head-wearable device 110and/or wrist-wearable device 170 and provide a user input via the userinterfaces and/or user interface elements to initiate the vestigialmuscle training mode. Alternatively or in addition, in some embodiments,the head-wearable device 110 automatically initiates the vestigialmuscle training mode when the head-wearable device 110 is worn by theuser 115. For example, for example, when the user 115 first wears thehead-wearable device 110, the head-wearable device 110 can initiate thevestigial muscle training mode and prompt the user 115 with an option toenable vestigial ear muscle detection.

In some embodiments, vestigial muscle training mode assesses the user'sability to move their ears via one or more vestigial ear muscles (e.g.,vestigial muscles associated with moving one's ears, such as ear 810).As described below, depending on the user's ability to move theirvestigial ear muscles, the vestigial muscle training mode can providesuggestions to the user 115 on how the user 115 can isolate andeffectively use different vestigial ear muscles to perform one or morevestigial ear muscle gestures. For example, the vestigial muscletraining mode can provide the user 115 with instructions to strengthenone or more vestigial ear muscles such that they are easier to identifyand/or move. This allows the user 115 learn one or more vestigial earmuscle gestures by using vestigial ear muscles as well as strengthentheir vestigial ear muscles such that performing vestigial ear musclegestures becomes second nature. Further, the vestigial muscle trainingmode can prompt the user 115 to move one or more vestigial ear musclesin a first direction and/or second direction to improve the range ofmotion of the vestigial ear muscle gesture. The vestigial muscletraining mode can be used to teach the user to move each earindependently and or both ears together.

For the purposes of training the user 115 to use their vestigial earmuscles, the systems and methods disclosed herein separate the vestigialear muscles into 4 zones around an ear. For example, ear 810 isassociated with the vestigial ear muscles in zone 1 802, zone 2 804,zone 3 806, and zone 4 808. While FIGS. 8A-8H illustrate training theuser 115 to perform vestigial ear muscle gestures using the vestigialear muscles in zone 2 802, the vestigial muscle training mode can trainthe user 115 to perform vestigial ear muscle gestures using anyvestigial ear muscles in zones 1 802, zone 2 802, zone 3 806, and zone 4808 and/or a combination thereof. Further, as the skilled artisan willappreciate upon reading the descriptions provided herein, the vestigialmuscles can be separated is more or less than four zones to train a userto perform vestigial ear muscle gestures.

In some embodiments, the head-wearable device 110 includes one or moresensors (e.g., EMG sensors as described below in reference to FIG. 11 )to capture the vestigial ear movements and facial muscle movementsperformed by the user 115. In some embodiments, the sensors are locatedwithin the head-wearable device 110 such that the sensors are adjacentto temporal, parietal, and/or occipital portions of the user 115's head.In this way, the sensors are in the closest proximity to the vestigialear muscles to sense the user 115's movements and obtain accurate data.In some embodiments, the sensors are integrated into the head-wearabledevice 110. Alternatively, in some embodiments, the sensors arephysically coupled to the head-wearable device 110 (e.g., coupled to anexterior surface), but operate independently of the head-wearable device110.

Turning to FIG. 8B, the head-wearable device 110 presents via a display130 (e.g., a heads-up display 130 b) a user interface and/or userinterface elements of the vestigial muscle training mode. For example,the head-wearable device 110 present to the user 115 a vestigial muscletraining mode prompt 812 that notifies the user 115 that the vestigialmuscle training mode is activate. The vestigial muscle training modeprompt 812 includes instructions to guide the user 115 through atraining process. For example, the vestigial muscle training mode prompt812 can instruct the user 115 to try moving the vestigial ear muscles inzone 2 804 by moving their nose 814 muscles. In some embodiments, tofurther guide the user 115, the vestigial muscle training mode prompt812 can present to the user 115 an outline of the one or more zones ofthe vestigial ear muscles, which provides a visual aid to the user 115to assist them in performing the suggested actions.

In some embodiments, the vestigial muscle training mode instructionspresented to the user 115, via the head-wearable device 110, train theuser 115 to target and use one or more vestigial ear muscles within oneor more zones. Alternatively, in some embodiments, the vestigial muscletraining mode instructions train the user 115 to target one or morevestigial ear muscles in one zone. The vestigial muscle training modecan teach the user 115 to identify vestigial ear muscles by instructingthe user 115 to perform one or more facial movements and/or move musclesclose to or connected with respective vestigial ear muscles such thatthe user 115 is able to feel or sense vestigial ear muscles within azone moving. For example, as shown in FIG. 8B, the vestigial muscletraining mode prompt 812 instructs the user to move their nose 814,which is a facial movement that can trigger or activate one or morevestigial ear muscles for performing a vestigial ear muscle gesture. Insome embodiments, facial muscles and/or muscle movements close to orconnected with vestigial ear muscles include the movement of any part ofthe face. Non-limiting examples of facial movements and/or musclemovements close to or connected with vestigial ear muscles includeeyebrows movement, nose movement, cheek movement, lip and/or mouthmovement, etc. Additionally, in some embodiments, the head-wearabledevice 110 can present or highlight one or more zones that are beingsensed by the one or more sensors as the user performs the instructionspresented while the vestigial muscle training mode is active.

In FIG. 8C, the user 115 performs the nose 814 movement suggested by thevestigial muscle training mode in order to move the vestigial earmuscles in zone 2 804. Even though the user 115 performs the suggestedfacial muscles and/or muscle movements close to or connected withvestigial ear muscles, the targeted vestigial ear muscles may not betriggered or activated (e.g., the sensor data obtained by thehead-wearable device 110 indicates that the one or more vestigial earmuscles were not moved). In such situations, the vestigial muscletraining mode can provide additional instructions to assist the user 115in identifying and/or moving the targeted vestigial ear muscles. Forexample, the performed nose 814 movement can generate facial movementrelated data, sensed by the sensors of the head-wearable device 110,that detects movement of vestigial ear muscles in a zone other than zone2 804 or does not detect any movement of vestigial ear muscles, andadditional instructions can be presented by the head-wearable device 110to further assist the user 115 in triggering or activating the vestigialear muscles in zone 2 804. Additionally, if the user 115 is unable toperform the suggested facial muscles and/or muscle movements close to orconnected with vestigial ear muscles, the vestigial muscle training modecan provide alternative instructions to assist the user 115 inidentifying and/or moving the targeted vestigial ear muscles.

FIG. 8D shows an example of additional or alternative instructionsprovided to the user 115 in accordance with a determination that theinitial instructions did not generate facial movement related data thatcan be used to detect movement of vestigial ear muscles in a targetedzone. For example, an updated vestigial muscle training mode prompt 822presented to the user 115 via the display 130 instructs the user 115 tomove their eyebrows 816 instead of their nose 114 in order to move thevestigial ear muscles in zone 2 804. In some embodiments, the user 115will continue to receive instructions on how to move the respectivevestigial ear muscles until they are successful. In some embodiments,the user 115 receives feedback on whether they are moving theirvestigial ear muscles correctly or incorrectly. During training, theuser 115 can select the method in which they prefer to receive thefeedback such as visual feedback on the display 130 (e.g., a guidedvideo, an outline of the targeted vestigial ear muscles and/or thecurrently moved muscles, etc.), haptic feedback (e.g., a vibrationidentifying the muscle to be moved), and/or auditory feedback (e.g.,audible instructions). Feedback can be provided through thehead-wearable device 110 or any other coupled device such as awrist-wearable device 170, an intermediary device, etc.

Turning to FIG. 8E, the user 115 is shown moving their eyebrows 816,which results in the user 115 successful activating the vestigial earmuscles in zone 2 804 (which is detected by the sensor of thehead-wearable device 110). As further shown in FIG. 8E, successfulactivation of the vestigial ear muscles in zone 2 804 also results inthe user 115 moving ear 810. As the user 115 becomes more accustomed tomoving targeted vestigial ear muscles, the vestigial muscle trainingmode can continue to provide additional instructions to further reduceexcess movements performed by the user 115. For example, after the user115 successfully activates the vestigial ear muscles in zone 2 804 andmoves their ear 810, the vestigial muscle training mode can provideadditional instructions to reduce or eliminate the facial musclemovements and/or muscle movements close to or connected with vestigialear muscles performed by the user 115 (e.g., the user's eyebrows 816 inFIG. 8E) to initially activates the vestigial ear muscles. In theexample shown in FIGS. 8E and 8F, the user 115 receives additionalinstructions to train the user 115 to move their ear 810 without usingadditional facial muscles and/or muscle movements close to or connectedwith vestigial ear muscles such as the user's nose 814 or eyebrows 816.

FIG. 8F illustrates an additional vestigial muscle training mode prompt832 presented on the display 130, which instructs the user 115 to trymoving the respective vestigial ear muscles in zone 2 804 without usingadditional facial muscles and/or muscle movements close to or connectedwith vestigial ear muscles. The vestigial muscle training mode providesinstructions to progressively assist the user 115 in identifying andisolating the targeted vestigial ear muscles. For example, in FIG. 8F,the user 115 was able to move the respective vestigial ear muscles inzone 2 804 and, as such, is further instructed to minimize facial musclemovements and/or muscle movements close to or connected with vestigialear muscles (e.g., as the user 115 is now familiar with the feeling andsensation a particular vestigial ear muscle movement).

FIG. 8G illustrates the user 115 successfully moving the vestigial earmuscles in zone 2 804 and moving their ear 810 without the use ofadditional facial muscle movements and/or muscle movements close to orconnected with vestigial ear muscles. In some embodiments, the user 115can record or store facial movement related data associated withvestigial ear muscle movement and assign it to a specific navigation orcontrol gesture (e.g., defining or creating a vestigial ear musclegesture). For example, as described above in reference to FIGS. 1E-1F,the user 115 can capture an image 160 using a vestigial ear musclegesture. In some embodiments, the user 115 can perform multiplevestigial ear muscle gestures (concurrently or sequentially) to performthe desired navigation or other controls. Additionally, the user 115 canalso record or store facial movement related data associated with one ormore vestigial ear muscle movements and assign them to differentcontrol/navigation gestures. Facial movement related data, in someembodiments, includes data associated with vestigial ear musclemovement, facial movement, muscle movements close to or connected withvestigial ear muscles, and/or any combination thereof.

FIG. 8H illustrates a successful vestigial muscle training mode prompt842 presented by the display 130 of the head-wearable device 110, whichnotifies the user 115 that they have successfully moved the vestigialear muscles in zone 2 804 independent of moving other facial muscles.The successful vestigial muscle training mode prompt 842 further informsthe user 115 that they can save their newly learned vestigial ear musclemovement to a specific action such as taking a picture, video, etc.(e.g., to define or create a vestigial ear muscle gesture).

FIG. 9 illustrates a detailed flow diagram of a method of detectingvestigial ear movements at a head-wearable device, according to someembodiments. Operations of the method 900 can be performed by one ormore processors of a head-wearable device 110. At least some of theoperations shown in FIG. 9 correspond to instructions stored in acomputer memory or computer-readable storage medium. Operations of themethod 900 can be performed by the head-wearable device 110 alone or inconjunction with one or more processors and/or hardware components ofanother device (e.g., a wrist-wearable device 170 and/or an intermediarydevice described below in reference to FIG. 11 ) communicatively coupledto the head-wearable device 110 and/or instructions stored in memory orcomputer-readable medium of the other device communicatively coupled tothe head-wearable device 110.

Method 900 includes being responsive (902) to an indication that a userwearing a head-wearable device requests to enable vestigial ear muscledetection and initiating a vestigial muscle training mode. As describedin FIG. 8A, the user 115 can manually enable the vestigial muscletraining mode or the vestigial training mode can be automaticallyactivated when the head-wearable device is first worn. In someembodiments, vestigial muscle training mode is used to train the user touse one or more vestigial ear muscles located in one or more zones asdescribed above in reference to FIGS. 8A-8H.

The method 900 includes while in a vestigial muscle training mode,presenting (904) instructions to the user to perform a facial movementto move a respective vestigial ear muscle of one or more vestigial earmuscles. In some embodiments, the instructions presented to the userinclude one or more facial movements connected to the respectivevestigial ear muscle. As described above in reference to FIGS. 8A-8H,the vestigial muscle training mode uses one or more facial movements toteach the user to move one or more vestigial ear muscles.

The method 900 further includes, in accordance with a determination(906) that facial movement related data (based on the facial movementperformed by the user), detected via sensors of the head-wearabledevice, satisfies detectable vestigial ear muscle movement criteriaassociated with movement of the respective vestigial ear muscle,providing (908) an indication that a vestigial ear muscle movement wasdetected. In some embodiments, as explained in FIGS. 8A, 8B, and 8C,during training the user can perform facial movements in order to movevestigial ear muscles connected to facial muscles. In some embodiments,facial movement related data is produced when the user moves either orboth facial muscles and/or vestigial ear muscles. The head-wearabledevice is able to use the facial movement related data to distinguishbetween facial movements and vestigial ear muscle movements in order todetermine whether the instructions provided to the user were able toassist the user in identifying and/or isolating vestigial ear muscles.In some embodiments, detectable vestigial ear muscle movement criteriais satisfied when the facial movement related data indicates that theuser is able to move their vestigial ear muscles with a predeterminedmagnitude (e.g., neuromuscular signals detected based on the vestigialear muscles are above a predetermined threshold (e.g., 50 μv, 1 mv,etc.), the user is able to move their ear a predetermined distance(e.g., 0.5 mm, 1 mm, 2 mm, etc.), the user is able to move their ear ata predetermined frequency, user is able to move their ear at apredetermined speed, etc. For example, if the user is only moving theirfacial muscles and not the vestigial ear muscles, the movements will notmeet the threshold for vestigial ear muscle movement criteria. Inanother example as illustrated in FIGS. 8C and 8D, the user can moveboth their facial muscles and their vestigial ear muscles to satisfy thedetectable vestigial ear muscle movement criteria.

In some embodiments, the indication shared with the user that thevestigial ear muscle movement was detected can be provided via auditory,haptic, or visual feedback. FIGS. 8C-8H illustrate example visualfeedback presented to the user via a display of the head-wearabledevice. In some embodiments, auditory, haptic, or visual feedback can beprovided via the head-wearable device or other coupled devices such as awrist-wearable device 170 or an intermediary device.

The method 900 further includes receiving (910) a user input to assign avestigial ear muscle gesture to the vestigial ear muscle movement. Insome embodiments, the user input can include an auditory input to thehead-wearable device or a coupled device, a physical input to a coupleddevice (e.g., a wrist-wearable device, smartphone, etc.), or anotherinput to the head-wearable device. In some embodiments, after the facialmovement related data corresponding to the vestigial ear muscle movementis determined to satisfy the detectable vestigial ear muscle movementcriteria, the user can assign a particular vestigial ear muscle gestureto the vestigial ear muscle movement (and/or the facial movement relateddata). The user can further associate the vestigial ear muscle gesturewith a particular control action or navigation action. For example, asshown in FIGS. 8A-8H, after learning to activate or move the vestigialear muscles in zone 2 804, the user can assign a vestigial ear musclegesture to the learned movement (e.g., capturing an image as describedabove in reference to FIGS. 1E-1F). In some embodiments, the user canperform one or more the vestigial ear muscle movements to create ordefine a vestigial ear muscle gesture associated with a particularaction or set of actions. More specifically, the vestigial ear musclegesture can be one or more vestigial ear muscle movements, a pattern ofone or more vestigial ear muscle movements, etc.

The method 900 further includes, storing (912) the facial movementrelated data associated with the vestigial ear muscle gesture. In otherwords, the vestigial ear muscle gesture and the associated data relatedto the vestigial ear muscle gesture can be stored such that the user canperform the vestigial ear muscle gesture to perform one or more controlactions or other actions via the head-wearable device.

FIG. 10 illustrates an example system for capturing and presentingcamera data as described above in reference to FIGS. 1A-9 . System 1000includes a wrist-wearable device 170 communicatively coupled with anintermediary device (such as a smartphone 1174 b) and a head-wearabledevice 110. In some embodiments, the wrist-wearable device 170, theintermediary device, and the head-wearable device 110 are connected viaa shared application. The shared application can facilitate the datatransfer between the different devices. For example, the sharedapplication can facilitate the transfer of sensor data, gesture data,camera data, instructions, etc.

Initially, one or more hand gestures are detected by the wrist-wearabledevice 170. The wrist-wearable device 170 can, upon detecting acamera-control gesture for capturing camera data, provide instructionsfor initiating the capture of camera data via a communicatively coupleddevice, such as the head-wearable device 110. In some embodiments, theinstructions are provided to the intermediary device for facilitatingthe capture of camera data at the desired device. For example, as shownin system 1000, the wrist-wearable device 170, upon detecting acamera-control gesture for capturing camera data, provides instructionsto the smartphone 1174 b (e.g., via the shared application) requestingan intent to capture camera data via the head-wearable device 110. Theinstructions provide by the wrist-wearable device 170 can specify thedevice that the user 115 would like to capture camera data from (e.g.,the smartphone 1174 b, the wrist-wearable device 170 itself, and/or anyother communicatively coupled device).

The head-wearable device 110 upon receiving instructions from theintermediary device begins to capture camera data (e.g., via its imagingdevice 111). The head-wearable device 110 after capturing camera datasends the captured camera data to the intermediary device (or the sharedapplication). The camera data sent to the intermediary device (or theshared application) is received at the wrist-wearable device 170, whichcan then present the captured camera data to the user 115 (e.g., via thedisplay of the wrist-wearable device 170). The captured camera data caninclude full images, full videos, thumbnails, clips or portions of thevideo, etc.

FIG. 11 illustrates a system 1100 of one or more devices for capturingand presenting camera data, in accordance with some embodiments. Forexample, a wrist-wearable device 170 is configured to detect one or morehand gestures performed by a user 115 and, based on a determination thata hand gesture performed by the user is a camera-control gesture, causeanother device (e.g., head-wearable devices 110 and 510) to capturecamera data via an imaging device 111, as well as cause the presentationof the camera data via a display 130. The system 1100 can include one ormore of servers 1170, electronic devices 1174 (e.g., a computer, 1174 a,a smartphone 1174 b, a controller 1174 c, and/or other devices),head-wearable devices 110, and/or wrist-wearable devices 170. In someembodiments, the one or more of servers 1170, electronic devices 1174,head-wearable devices 110, and/or wrist-wearable devices 170 arecommunicatively coupled via a network 1172. In some embodiments, thewrist-wearable device 170 is configured to cause one or more operationsto be performed by a communicatively coupled head-wearable device 110,and/or the two devices can also both be connected to an intermediarydevice, such as a smartphone 1174 b, a controller 1174 c, or otherdevice that provides instructions and data to and between the twodevices. In some embodiments, the wrist-wearable device 170 isconfigured to cause one or more operations to be performed by multipledevices in conjunction with the head-wearable device 110. In someembodiments, instructions to cause the performance of one or moreoperations are controlled via an artificial reality (AR) processingmodule 1145. The AR processing module 1145 can be implemented in one ormore devices, such as the one or more of servers 1170, electronicdevices 1174, head-wearable devices 110, and/or wrist-wearable devices170. In some embodiments, the one or more devices perform operations ofthe AR processing module 1145, using one or more respective processors,individually or in conjunction with at least one other device asdescribed herein.

In some embodiments, the wrist-wearable device 170 includes one or morecomponents such as a communication interface 1115 a, one or more sensors1121 a, one or more haptic generators 1125 a, an AR processing module1145 a, one or more imaging devices 111 a (e.g., a camera), one or moreprocessors 1150 a, and memory 1160 a. In addition, in some embodiments,the wrist-wearable device 170 includes a display 130 a and one or moreapplications 1135 a. In some embodiments, the memory 1160 a isconfigured to store sensor data 1164 a and AR processing data 1163 a.Although not show, in some embodiments, the memory 1160 a can includeapplication data, device data (e.g., device hardware, device model,etc.), image data, and/or user data (e.g., data collected through use ofa device, data collected through use of an application, userpreferences, or other information stored by the user). In someembodiments, the one or more components of the wrist-wearable device 170are housed within a capsule (or watch body) and/or a band of thewrist-wearable device 170.

In some embodiments, the communications interface 1115 (e.g., 1115 a,1115 b, and 1115 c) is configured to communicatively couple thewrist-wearable device 170 to one or more other devices such as thehead-wearable device 110, electronic device 1174 (e.g., a computer 1174a, a smartphone 1174 b, a controller 1174 c, a tablet, etc.), and/or oneor more servers 1170. The communication interface 1115 is used establishwired or wireless connections between the wrist-wearable device 170 andthe other devices. In some embodiments, the communication interface 1115includes hardware capable of data communications using any of a varietyof custom or standard wireless protocols (e.g., IEEE 802.15.4, Wi-Fi,ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a,WirelessHART, or MiWi), custom or standard wired protocols (e.g.,Ethernet or HomePlug), and/or any other suitable communication protocol.

The one or more sensors 1121 (e.g., 1121 a, 1121 b, and 1121 c) caninclude heart rate sensors, neuromuscular-signal sensors (e.g.,electromyography (EMG) sensors), SpO2 sensors, altimeters, thermalsensors or thermal couples, ambient light sensors, ambient noisesensors, and/or inertial measurement units (IMU)s. Additionalnon-limiting examples of the one or more sensors 1121 include, e.g.,infrared, pyroelectric, ultrasonic, microphone, laser, optical, Doppler,gyro, accelerometer, resonant LC sensors, capacitive sensors, acousticsensors, and/or inductive sensors. In some embodiments, the one or moresensors 1121 are configured to gather additional data about the user(e.g., an impedance of the user's body). Examples of sensor data outputby these sensors includes body temperature data, infrared range-finderdata, positional information, motion data, activity recognition data,silhouette detection and recognition data, gesture data, heart ratedata, and other wearable device data (e.g., biometric readings andoutput, accelerometer data). The one or more sensors 1121 can includelocation sensing devices (e.g., GPS) configured to provide locationinformation. In some embodiment, the data measured or sensed by the oneor more sensors 1121 is stored in memory 1160. In some embodiments, thesensor data is used by the wrist-wearable device 170 for detecting oneor more hand gestures and determining one or more control gesture types,such as a camera-control gesture.

The one or more haptic generators 1125 (e.g., 1125 a and 1125 b) caninclude one or more actuators (e.g., eccentric rotating mass (ERM),linear resonant actuators (LRA), voice coil motor (VCM), piezo hapticactuator, thermoelectric devices, solenoid actuators, ultrasonictransducers or sensors, etc.). In some embodiments, the one or morehaptic generators 1125 are hydraulic, pneumatic, electric, and/ormechanical actuators. In some embodiments, the one or more hapticgenerators 1125 are part of a surface of the wrist-wearable device 170that can be used to generate a haptic response (e.g., a thermal changeat the surface, a tightening or loosening of a band, increase ordecrease in pressure, etc.). For example, the one or more hapticgenerators 1125 can apply vibration stimulations, pressure stimulations,squeeze simulations, shear stimulations, temperature changes, or somecombination thereof to the user. In addition, in some embodiments, theone or more haptic generators 1125 include audio generating devices(e.g., speakers and other sound transducers) and illuminating devices(e.g., light-emitting diodes (LED)s, screen displays, etc.). The one ormore haptic generators 1125 can be used to generate different audiblesounds and/or visible lights that are provided to the user as hapticresponses. The above list of haptic generators is non-exhaustive; anyaffective devices can be used to generate one or more haptic responsesthat are delivered to a user.

In some embodiments, the one or more applications 1135 (e.g., 1135 a,1135 b, and 1135 c) include social-media applications, bankingapplications, health applications, messaging applications, web browsers,gaming application, streaming applications, media applications, imagingapplications, productivity applications, social applications, etc. Insome embodiments, the one or more applications 1135 include artificialreality applications. The one or more applications 1135 can beconfigured to provide data to the head-wearable device 110 that can beused to determine variable light-based representations. In someembodiments, the one or more applications 1135 can be displayed via anoptional display of the head-wearable device 110.

In some embodiments, the AR processing module 1145 (e.g., 1145 a, 1145b, and 1145 c) is configured to detect and determine one or moregestures performed by the user 115 based at least on sensor data. Insome embodiments, the AR processing module 1145 is configured detect anddetermine one or more gestures performed by the user 115 based on cameradata received that captures at least a portion of the user 115's hand.For example, the wrist-wearable device 170 can receive EMG data and/orIMU data from one or more sensors 1121 based on the user 115'sperformance of a hand gesture and provide the sensor data to the ARprocessing module 1145 for gesture detection and identification. The ARprocessing module 1145, based on the detection and determination of agesture, causes a device communicatively coupled to the wrist-wearabledevice 170 to perform an operation (or action). For example, a handgesture performed by the user and determined to be a camera-controlgesture, based on the sensor data received at the wrist-wearable device170, causes a head-wearable device 110 or 510 to capture image data aswell as the presentation of a representation of the image data at acoupled display (e.g., display 130). In some embodiments, the ARprocessing module 1145 determines different characteristics between thehand gestures performed by the user based on the sensor data, andperforms one or more operations or actions based on the characteristicsof the hand gesture satisfying one or more thresholds (e.g., forcethresholds, duration thresholds, contact number thresholds, etc.).

In some embodiments, the one or more imaging devices 111 (e.g., 111 aand 111 b) can include an ultra-wide camera, a wide camera, a telephotocamera, a depth-sensing cameras, or other types of cameras. In someembodiments, the one or more imaging devices 111 are used to captureimage data and/or video data via the wrist-wearable device 170. Thecaptured image data can be processed and stored in memory and thenpresented to a user for viewing. The one or more imaging devices 111 caninclude one or more modes for capturing image data or video data. Forexample, these modes can include a high-dynamic range (HDR) imagecapture mode, a low light image capture mode, burst image capture mode,and other modes. In some embodiments, a particular mode is automaticallyselected based on the environment (e.g., lighting, movement of thedevice, etc.). For example, a wrist-wearable device with HDR imagecapture mode and a low light image capture mode active can automaticallyselect the appropriate mode based on the environment (e.g., darklighting may result in the use of low light image capture mode insteadof HDR image capture mode). In some embodiments, the user can select themode. The image data and/or video data captured by the one or moreimaging devices 111 is stored in memory 1160 (which can include volatileand non-volatile memory such that the image data and/or video data canbe temporarily or permanently stored, as needed depending on thecircumstances).

The one or more processors 1150 (e.g., 1150 a, 1150 b, and 1150 c) canbe implemented as any kind of computing device, such as an integratedsystem-on-a-chip, a microcontroller, a fixed programmable gate array(FPGA), a microprocessor, and/or other application specific integratedcircuits (ASICs). The processor may operate in conjunction with memory1160. The memory 1160 may be or include random access memory (RAM),read-only memory (ROM), dynamic random access memory (DRAM), staticrandom access memory (SRAM) and magnetoresistive random access memory(MRAM), and may include firmware, such as static data or fixedinstructions, basic input/output system (BIOS), system functions,configuration data, and other routines used during the operation of thewrist-wearable device 170 and the processor 1150. The memory 1160 alsoprovides a storage area for data and instructions associated withapplications and data handled by the processor 1150.

In some embodiments, the memory 1160 (e.g., 1160 a, 1160 b, and 1160 c)stores at least user data 1162 including sensor data 1164 and ARprocessing data 1163. The sensor data 1164 includes sensor datamonitored by one or more sensors 1121 of the wrist-wearable device 170and/or sensor data received from one or more devices communicativecoupled with the wrist-wearable device 170, such as a head-wearabledevice 110, smartphone 1174 b, etc. The sensor data 1164 can includesensor data collected over a predetermined period of time that can beused by the AR processing module 1145. The AR processing data 1163 caninclude one or more one or more predefined camera-control gestures, userdefined camera-control gestures, predefined non-camera-control gestures,and/or user defined non-camera-control gestures. In some embodiments,the AR processing data 1163 further includes one or more predeterminedthreshold for different gestures.

The head-wearable devices 110 and 510 can include a communicationinterface 1115 a, a display 130 b, one or more sensors 1121 b, one ormore haptic generators 1125 b, one or more imaging devices 111 b (e.g.,a camera), one or more applications 1135 b, one or more processors 1150b, and memory 1160 b. In some embodiments, the wrist-wearable device 170includes smart glasses (e.g., the augmented-reality glasses), artificialreality headsets (e.g., VR/AR headsets), or other head worn device. Insome embodiments, one or more components of the head-wearable device 110are housed within a body of the head-wearable device 110 (e.g., framesof smart glasses, a body of an AR headset, etc.). In addition, in someembodiments, one or more components of the head-wearable device 110 arestored within or coupled with lenses of the head-wearable device 110.The wrist-wearable device 170 is configured to communicatively couplewith the head-wearable devices 110 and/or 510 (or other devices (e.g.,electronic device 1174)) using communication interface 1115, 1115 a,and/or 1115 b. In some embodiments, the wrist-wearable device 170 isconfigured to communicatively couple with the head-wearable device 110(or other devices (e.g., electronic device 1174)) via an applicationprogramming interface (API). In some embodiments, the wrist-wearabledevice 170 operates in conjunction with the head-wearable device 110 toperform one or more operations, such as capturing camera data andpresenting a representation of the image data at a coupled display.Similar to the wrist-wearable device 170, the head-wearable device 110can use the AR processing module 1145 b to determine and cause theperformance of one or more hand gestures, such as camera-controlgestures.

Electronic devices 1174 can also include a communication interface 1115c, a display 130 c, one or more sensors 1121 c, one or more applications1135 c, an AR processing module 1145 c, one or more processors 1150 c,and memory 1160 c. The electronic devices 1174 are configured tocommunicatively couple with the wrist-wearable device 170 and/orhead-wearable device 110 (or other devices) using communicationinterface 1115, 1115 a, 1115 b, and/or 1115 c. In some embodiments, theelectronic devices 1174 are configured to communicatively couple withthe wrist-wearable device 170 and/or head-wearable device 110 (or otherdevices) via an application programming interface (API). In someembodiments, the electronic devices 1174 operate in conjunction with thewrist-wearable device 170 and/or the head-wearable device 110 todetermine a hand gesture and cause the performance of an operation oraction at a communicatively coupled device. The electronic devices 1174,like the head-wearable device 110.

Server 1170 includes a communication interface 1115, one or moreapplications 1135, an AR processing module 1145, one or more processors1150, and memory 1160. In some embodiments, the server 1170 isconfigured to receive sensor data from one or more devices, such as thehead-wearable device 110, the wrist-wearable device 170, and/orelectronic device 1174, and use the received sensor data to determine ahand gesture. The server 1170 can generate instructions that cause theperformance of operations and actions associated with a determined handgesture at communicatively coupled devices, such as the head-wearabledevice 110.

Further embodiments also include various subsets of the aboveembodiments including embodiments described with reference to FIGS.1A-8H combined or otherwise re-arranged.

Example Wrist-Wearable Devices

FIGS. 12A and 12B illustrate an example wrist-wearable device 1250, inaccordance with some embodiments. The wrist-wearable device 1250 is aninstance of the wearable device described herein, such that the wearabledevice should be understood to have the features of the wrist-wearabledevice 1250 and vice versa. FIG. 12A illustrates a perspective view ofthe wrist-wearable device 1250 that includes a watch body 1254 coupledwith a watch band 1262. The watch body 1254 and the watch band 1262 canhave a substantially rectangular or circular shape and can be configuredto allow a user to wear the wrist-wearable device 1250 on a body part(e.g., a wrist). The wrist-wearable device 1250 can include a retainingmechanism 1267 (e.g., a buckle, a hook and loop fastener, etc.) forsecuring the watch band 1262 to the user's wrist. The wrist-wearabledevice 1250 can also include a coupling mechanism 1260 (e.g., a cradle)for detachably coupling the capsule or watch body 1254 (via a couplingsurface of the watch body 1254) to the watch band 1262.

The wrist-wearable device 1250 can perform various functions associatedwith navigating through user interfaces and selectively openingapplications, as described above with reference to Figures . . .[replace with citation to UI Figures related to watch]. As will bedescribed in more detail below, operations executed by thewrist-wearable device 1250 can include, without limitation, display ofvisual content to the user (e.g., visual content displayed on display1256); sensing user input (e.g., sensing a touch on peripheral button1268, sensing biometric data on sensor 1264, sensing neuromuscularsignals on neuromuscular sensor 1265, etc.); messaging (e.g., text,speech, video, etc.); image capture; wireless communications (e.g.,cellular, near field, Wi-Fi, personal area network, etc.); locationdetermination; financial transactions; providing haptic feedback;alarms; notifications; biometric authentication; health monitoring;sleep monitoring; etc. These functions can be executed independently inthe watch body 1254, independently in the watch band 1262, and/or incommunication between the watch body 1254 and the watch band 1262. Insome embodiments, functions can be executed on the wrist-wearable device1250 in conjunction with an artificial-reality environment thatincludes, but is not limited to, virtual-reality (VR) environments(including non-immersive, semi-immersive, and fully immersive VRenvironments); augmented-reality environments (including marker-basedaugmented-reality environments, markerless augmented-realityenvironments, location-based augmented-reality environments, andprojection-based augmented-reality environments); hybrid reality; andother types of mixed-reality environments. As the skilled artisan willappreciate upon reading the descriptions provided herein, the novelwearable devices described herein can be used with any of these types ofartificial-reality environments.

The watch band 1262 can be configured to be worn by a user such that aninner surface of the watch band 1262 is in contact with the user's skin.When worn by a user, sensor 1264 is in contact with the user's skin. Thesensor 1264 can be a biosensor that senses a user's heart rate,saturated oxygen level, temperature, sweat level, muscle intentions, ora combination thereof. The watch band 1262 can include multiple sensors1264 that can be distributed on an inside and/or an outside surface ofthe watch band 1262. Additionally, or alternatively, the watch body 1254can include sensors that are the same or different than those of thewatch band 1262 (or the watch band 1262 can include no sensors at all insome embodiments). For example, multiple sensors can be distributed onan inside and/or an outside surface of the watch body 1254. As describedbelow with reference to FIGS. 12B and/or 12C, the watch body 1254 caninclude, without limitation, a front-facing image sensor 1225A and/or arear-facing image sensor 1225B, a biometric sensor, an IMU, a heart ratesensor, a saturated oxygen sensor, a neuromuscular sensor(s), analtimeter sensor, a temperature sensor, a bioimpedance sensor, apedometer sensor, an optical sensor (e.g., imaging sensor 12104), atouch sensor, a sweat sensor, etc. The sensor 1264 can also include asensor that provides data about a user's environment including a user'smotion (e.g., an IMU), altitude, location, orientation, gait, or acombination thereof. The sensor 1264 can also include a light sensor(e.g., an infrared light sensor, a visible light sensor) that isconfigured to track a position and/or motion of the watch body 1254and/or the watch band 1262. The watch band 1262 can transmit the dataacquired by sensor 1264 to the watch body 1254 using a wiredcommunication method (e.g., a Universal AsynchronousReceiver/Transmitter (UART), a USB transceiver, etc.) and/or a wirelesscommunication method (e.g., near field communication, Bluetooth, etc.).The watch band 1262 can be configured to operate (e.g., to collect datausing sensor 1264) independent of whether the watch body 1254 is coupledto or decoupled from watch band 1262.

In some examples, the watch band 1262 can include a neuromuscular sensor1265 (e.g., an EMG sensor, a mechanomyogram (MMG) sensor, asonomyography (SMG) sensor, etc.). Neuromuscular sensor 1265 can sense auser's intention to perform certain motor actions. The sensed muscleintention can be used to control certain user interfaces displayed onthe display 1256 of the wrist-wearable device 1250 and/or can betransmitted to a device responsible for rendering an artificial-realityenvironment (e.g., a head-mounted display) to perform an action in anassociated artificial-reality environment, such as to control the motionof a virtual device displayed to the user.

Signals from neuromuscular sensor 1265 can be used to provide a userwith an enhanced interaction with a physical object and/or a virtualobject in an artificial-reality application generated by anartificial-reality system (e.g., user interface objects presented on thedisplay 1256, or another computing device (e.g., a smartphone)). Signalsfrom neuromuscular sensor 1265 can be obtained (e.g., sensed andrecorded) by one or more neuromuscular sensors 1265 of the watch band1262. Although FIG. 12A shows one neuromuscular sensor 1265, the watchband 1262 can include a plurality of neuromuscular sensors 1265 arrangedcircumferentially on an inside surface of the watch band 1262 such thatthe plurality of neuromuscular sensors 1265 contact the skin of theuser. The watch band 1262 can include a plurality of neuromuscularsensors 1265 arranged circumferentially on an inside surface of thewatch band 1262. Neuromuscular sensor 1265 can sense and recordneuromuscular signals from the user as the user performs muscularactivations (e.g., movements, gestures, etc.). The muscular activationsperformed by the user can include static gestures, such as placing theuser's hand palm down on a table; dynamic gestures, such as grasping aphysical or virtual object; and covert gestures that are imperceptibleto another person, such as slightly tensing a joint by co-contractingopposing muscles or using sub-muscular activations. The muscularactivations performed by the user can include symbolic gestures (e.g.,gestures mapped to other gestures, interactions, or commands, forexample, based on a gesture vocabulary that specifies the mapping ofgestures to commands).

The watch band 1262 and/or watch body 1254 can include a haptic device1263 (e.g., a vibratory haptic actuator) that is configured to providehaptic feedback (e.g., a cutaneous and/or kinesthetic sensation, etc.)to the user's skin. The sensors 1264 and 1265, and/or the haptic device1263 can be configured to operate in conjunction with multipleapplications including, without limitation, health monitoring, socialmedia, game playing, and artificial reality (e.g., the applicationsassociated with artificial reality).

The wrist-wearable device 1250 can include a coupling mechanism (alsoreferred to as a cradle) for detachably coupling the watch body 1254 tothe watch band 1262. A user can detach the watch body 1254 from thewatch band 1262 in order to reduce the encumbrance of the wrist-wearabledevice 1250 to the user. The wrist-wearable device 1250 can include acoupling surface on the watch body 1254 and/or coupling mechanism(s)1260 (e.g., a cradle, a tracker band, a support base, a clasp). A usercan perform any type of motion to couple the watch body 1254 to thewatch band 1262 and to decouple the watch body 1254 from the watch band1262. For example, a user can twist, slide, turn, push, pull, or rotatethe watch body 1254 relative to the watch band 1262, or a combinationthereof, to attach the watch body 1254 to the watch band 1262 and todetach the watch body 1254 from the watch band 1262.

As shown in the example of FIG. 12A, the watch band coupling mechanism1260 can include a type of frame or shell that allows the watch body1254 coupling surface to be retained within the watch band couplingmechanism 1260. The watch body 1254 can be detachably coupled to thewatch band 1262 through a friction fit, magnetic coupling, arotation-based connector, a shear-pin coupler, a retention spring, oneor more magnets, a clip, a pin shaft, a hook and loop fastener, or acombination thereof. In some examples, the watch body 1254 can bedecoupled from the watch band 1262 by actuation of the release mechanism1270. The release mechanism 1270 can include, without limitation, abutton, a knob, a plunger, a handle, a lever, a fastener, a clasp, adial, a latch, or a combination thereof.

As shown in FIGS. 12A-12B, the coupling mechanism 1260 can be configuredto receive a coupling surface proximate to the bottom side of the watchbody 1254 (e.g., a side opposite to a front side of the watch body 1254where the display 1256 is located), such that a user can push the watchbody 1254 downward into the coupling mechanism 1260 to attach the watchbody 1254 to the coupling mechanism 1260. In some embodiments, thecoupling mechanism 1260 can be configured to receive a top side of thewatch body 1254 (e.g., a side proximate to the front side of the watchbody 1254 where the display 1256 is located) that is pushed upward intothe cradle, as opposed to being pushed downward into the couplingmechanism 1260. In some embodiments, the coupling mechanism 1260 is anintegrated component of the watch band 1262 such that the watch band1262 and the coupling mechanism 1260 are a single unitary structure.

The wrist-wearable device 1250 can include a single release mechanism1270 or multiple release mechanisms 1270 (e.g., two release mechanisms1270 positioned on opposing sides of the wrist-wearable device 1250 suchas spring-loaded buttons). As shown in FIG. 12A, the release mechanism1270 can be positioned on the watch body 1254 and/or the watch bandcoupling mechanism 1260. Although FIG. 12A shows release mechanism 1270positioned at a corner of watch body 1254 and at a corner of watch bandcoupling mechanism 1260, the release mechanism 1270 can be positionedanywhere on watch body 1254 and/or watch band coupling mechanism 1260that is convenient for a user of wrist-wearable device 1250 to actuate.A user of the wrist-wearable device 1250 can actuate the releasemechanism 1270 by pushing, turning, lifting, depressing, shifting, orperforming other actions on the release mechanism 1270. Actuation of therelease mechanism 1270 can release (e.g., decouple) the watch body 1254from the watch band coupling mechanism 1260 and the watch band 1262allowing the user to use the watch body 1254 independently from watchband 1262. For example, decoupling the watch body 1254 from the watchband 1262 can allow the user to capture images using rear-facing imagesensor 1225B.

FIG. 12B includes top views of examples of the wrist-wearable device1250. The examples of the wrist-wearable device 1250 shown in FIGS.12A-12B can include a coupling mechanism 1260 (as shown in FIG. 12B, theshape of the coupling mechanism can correspond to the shape of the watchbody 1254 of the wrist-wearable device 1250). The watch body 1254 can bedetachably coupled to the coupling mechanism 1260 through a frictionfit, magnetic coupling, a rotation-based connector, a shear-pin coupler,a retention spring, one or more magnets, a clip, a pin shaft, a hook andloop fastener, or any combination thereof.

In some examples, the watch body 1254 can be decoupled from the couplingmechanism 1260 by actuation of a release mechanism 1270. The releasemechanism 1270 can include, without limitation, a button, a knob, aplunger, a handle, a lever, a fastener, a clasp, a dial, a latch, or acombination thereof. In some examples, the wristband system functionscan be executed independently in the watch body 1254, independently inthe coupling mechanism 1260, and/or in communication between the watchbody 1254 and the coupling mechanism 1260. The coupling mechanism 1260can be configured to operate independently (e.g., execute functionsindependently) from watch body 1254. Additionally, or alternatively, thewatch body 1254 can be configured to operate independently (e.g.,execute functions independently) from the coupling mechanism 1260. Asdescribed below with reference to the block diagram of FIG. 12A, thecoupling mechanism 1260 and/or the watch body 1254 can each include theindependent resources required to independently execute functions. Forexample, the coupling mechanism 1260 and/or the watch body 1254 can eachinclude a power source (e.g., a battery), a memory, data storage, aprocessor (e.g., a central processing unit (CPU)), communications, alight source, and/or input/output devices.

The wrist-wearable device 1250 can have various peripheral buttons 1272,1274, and 1276, for performing various operations at the wrist-wearabledevice 1250. Also, various sensors, including one or both of the sensors1264 and 1265, can be located on the bottom of the watch body 1254, andcan optionally be used even when the watch body 1254 is detached fromthe watch band 1262.

FIG. 12C is a block diagram of a computing system 12000, according to atleast one embodiment of the present disclosure. The computing system12000 includes an electronic device 12002, which can be, for example, awrist-wearable device. The wrist-wearable device 1250 described indetail above with respect to FIGS. 12A-12B is an example of theelectronic device 12002, so the electronic device 12002 will beunderstood to include the components shown and described below for thecomputing system 12000. In some embodiments, all, or a substantialportion of the components of the computing system 12000 are included ina single integrated circuit. In some embodiments, the computing system12000 can have a split architecture (e.g., a split mechanicalarchitecture, a split electrical architecture) between a watch body(e.g., a watch body 1254 in FIGS. 12A-12B) and a watch band (e.g., awatch band 1262 in FIGS. 12A-12B). The electronic device 12002 caninclude a processor (e.g., a central processing unit 12004), acontroller 12010, a peripherals interface 12014 that includes one ormore sensors 12100 and various peripheral devices, a power source (e.g.,a power system 12300), and memory (e.g., a memory 12400) that includesan operating system (e.g., an operating system 12402), data (e.g., data12410), and one or more applications (e.g., applications 12430).

In some embodiments, the computing system 12000 includes the powersystem 12300 which includes a charger input 12302, a power-managementintegrated circuit (PMIC) 12304, and a battery 12306.

In some embodiments, a watch body and a watch band can each beelectronic devices 12002 that each have respective batteries (e.g.,battery 12306), and can share power with each other. The watch body andthe watch band can receive a charge using a variety of techniques. Insome embodiments, the watch body and the watch band can use a wiredcharging assembly (e.g., power cords) to receive the charge.Alternatively, or in addition, the watch body and/or the watch band canbe configured for wireless charging. For example, a portable chargingdevice can be designed to mate with a portion of watch body and/or watchband and wirelessly deliver usable power to a battery of watch bodyand/or watch band.

The watch body and the watch band can have independent power systems12300 to enable each to operate independently. The watch body and watchband can also share power (e.g., one can charge the other) viarespective PMICs 12304 that can share power over power and groundconductors and/or over wireless charging antennas.

In some embodiments, the peripherals interface 12014 can include one ormore sensors 12100. The sensors 12100 can include a coupling sensor12102 for detecting when the electronic device 12002 is coupled withanother electronic device 12002 (e.g., a watch body can detect when itis coupled to a watch band, and vice versa). The sensors 12100 caninclude imaging sensors 12104 for collecting imaging data, which canoptionally be the same device as one or more of the cameras 12218. Insome embodiments, the imaging sensors 12104 can be separate from thecameras 12218. In some embodiments the sensors include an SpO2 sensor12106. In some embodiments, the sensors 12100 include an EMG sensor12108 for detecting, for example muscular movements by a user of theelectronic device 12002. In some embodiments, the sensors 12100 includea capacitive sensor 12110 for detecting changes in potential of aportion of a user's body. In some embodiments, the sensors 12100 includea heart rate sensor 12112. In some embodiments, the sensors 12100include an inertial measurement unit (IMU) sensor 12114 for detecting,for example, changes in acceleration of the user's hand.

In some embodiments, the peripherals interface 12014 includes anear-field communication (NFC) component 12202, a global-position system(GPS) component 12204, a long-term evolution (LTE) component 12206, andor a Wi-Fi or Bluetooth communication component 12208.

In some embodiments, the peripherals interface includes one or morebuttons (e.g., the peripheral buttons 1272, 1274, and 1276 in FIG. 12B),which, when selected by a user, cause operation to be performed at theelectronic device 12002.

The electronic device 12002 can include at least one display 12212, fordisplaying visual affordances to the user, including user-interfaceelements and/or three-dimensional virtual objects. The display can alsoinclude a touch screen for inputting user inputs, such as touchgestures, swipe gestures, and the like.

The electronic device 12002 can include at least one speaker 12214 andat least one microphone 12216 for providing audio signals to the userand receiving audio input from the user. The user can provide userinputs through the microphone 12216 and can also receive audio outputfrom the speaker 12214 as part of a haptic event provided by the hapticcontroller 12012.

The electronic device 12002 can include at least one camera 12218,including a front camera 12220 and a rear camera 12222. In someembodiments, the electronic device 12002 can be a head-wearable device,and one of the cameras 12218 can be integrated with a lens assembly ofthe head-wearable device.

One or more of the electronic devices 12002 can include one or morehaptic controllers 12012 and associated componentry for providing hapticevents at one or more of the electronic devices 12002 (e.g., a vibratingsensation or audio output in response to an event at the electronicdevice 12002). The haptic controllers 12012 can communicate with one ormore electroacoustic devices, including a speaker of the one or morespeakers 12214 and/or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). The haptic controller 12012 can provide haptic events to thatare capable of being sensed by a user of the electronic devices 12002.In some embodiments, the one or more haptic controllers 12012 canreceive input signals from an application of the applications 12430.

Memory 12400 optionally includes high-speed random-access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Access to the memory 12400 byother components of the electronic device 12002, such as the one or moreprocessors of the central processing unit 12004, and the peripheralsinterface 12014 is optionally controlled by a memory controller of thecontrollers 12010.

In some embodiments, software components stored in the memory 12400 caninclude one or more operating systems 12402 (e.g., a Linux-basedoperating system, an Android operating system, etc.). The memory 12400can also include data 12410, including structured data (e.g., SQLdatabases, MongoDB databases, GraphQL data, JSON data, etc.). The data12410 can include profile data 12412, sensor data 12414, media file data12414.

In some embodiments, software components stored in the memory 12400include one or more applications 12430 configured to be performoperations at the electronic devices 12002. In some embodiments, the oneor more applications 12430 include one or more communication interfacemodules 12432, one or more graphics modules 12434, one or more cameraapplication modules 12436. In some embodiments, a plurality ofapplications 12430 can work in conjunction with one another to performvarious tasks at one or more of the electronic devices 12002.

It should be appreciated that the electronic devices 12002 are only someexamples of the electronic devices 12002 within the computing system12000, and that other electronic devices 12002 that are part of thecomputing system 12000 can have more or fewer components than shownoptionally combines two or more components, or optionally have adifferent configuration or arrangement of the components. The variouscomponents shown in FIG. 12C are implemented in hardware, software,firmware, or a combination thereof, including one or more signalprocessing and/or application-specific integrated circuits.

As illustrated by the lower portion of FIG. 12C, various individualcomponents of a wrist-wearable device can be examples of the electronicdevice 12002. For example, some or all of the components shown in theelectronic device 12002 can be housed or otherwise disposed in acombined watch device 12002A, or within individual components of thecapsule device watch body 12002B, the cradle portion 12002C, and/or awatch band.

FIG. 12D illustrates a wearable device 12170, in accordance with someembodiments. In some embodiments, the wearable device 12170 is used togenerate control information (e.g., sensed data about neuromuscularsignals or instructions to perform certain commands after the data issensed) for causing a computing device to perform one or more inputcommands. In some embodiments, the wearable device 12170 includes aplurality of neuromuscular sensors 12176. In some embodiments, theplurality of neuromuscular sensors 12176 includes a predetermined numberof (e.g., 16) neuromuscular sensors (e.g., EMG sensors) arrangedcircumferentially around an elastic band 12174. The plurality ofneuromuscular sensors 12176 may include any suitable number ofneuromuscular sensors. In some embodiments, the number and arrangementof neuromuscular sensors 12176 depends on the particular application forwhich the wearable device 12170 is used. For instance, a wearable device12170 configured as an armband, wristband, or chest-band may include aplurality of neuromuscular sensors 12176 with different number ofneuromuscular sensors and different arrangement for each use case, suchas medical use cases as compared to gaming or general day-to-day usecases. For example, at least 16 neuromuscular sensors 12176 may bearranged circumferentially around elastic band 12174.

In some embodiments, the elastic band 12174 is configured to be wornaround a user's lower arm or wrist. The elastic band 12174 may include aflexible electronic connector 12172. In some embodiments, the flexibleelectronic connector 12172 interconnects separate sensors and electroniccircuitry that are enclosed in one or more sensor housings.Alternatively, in some embodiments, the flexible electronic connector12172 interconnects separate sensors and electronic circuitry that areoutside of the one or more sensor housings. Each neuromuscular sensor ofthe plurality of neuromuscular sensors 12176 can include askin-contacting surface that includes one or more electrodes. One ormore sensors of the plurality of neuromuscular sensors 12176 can becoupled together using flexible electronics incorporated into thewearable device 12170. In some embodiments, one or more sensors of theplurality of neuromuscular sensors 12176 can be integrated into a wovenfabric, wherein the fabric one or more sensors of the plurality ofneuromuscular sensors 12176 are sewn into the fabric and mimic thepliability of fabric (e.g., the one or more sensors of the plurality ofneuromuscular sensors 12176 can be constructed from a series wovenstrands of fabric). In some embodiments, the sensors are flush with thesurface of the textile and are indistinguishable from the textile whenworn by the user.

FIG. 12E illustrates a wearable device 12179 in accordance with someembodiments. The wearable device 12179 includes paired sensor channels12185 a-12185 f along an interior surface of a wearable structure 12175that are configured to detect neuromuscular signals. Different number ofpaired sensors channels can be used (e.g., one pair of sensors, threepairs of sensors, four pairs of sensors, or six pairs of sensors). Thewearable structure 12175 can include a band portion 12190, a capsuleportion 12195, and a cradle portion (not pictured) that is coupled withthe band portion 12190 to allow for the capsule portion 12195 to beremovably coupled with the band portion 12190. For embodiments in whichthe capsule portion 12195 is removable, the capsule portion 12195 can bereferred to as a removable structure, such that in these embodiments thewearable device includes a wearable portion (e.g., band portion 12190and the cradle portion) and a removable structure (the removable capsuleportion which can be removed from the cradle). In some embodiments, thecapsule portion 12195 includes the one or more processors and/or othercomponents of the wrist-wearable device 170 described above in referenceto FIG. 11 . The wearable structure 12175 is configured to be worn by auser 115. More specifically, the wearable structure 12175 is configuredto couple the wearable device 12179 to a wrist, arm, forearm, or otherportion of the user's body. Each paired sensor channels 12185 a-12185 fincludes two electrodes 12180 (e.g., electrodes 12180 a-12180 h) forsensing neuromuscular signals based on differential sensing within eachrespective sensor channel. In accordance with some embodiments, thewearable device 12170 further includes an electrical ground and ashielding electrode.

The techniques described above can be used with any device for sensingneuromuscular signals, including the arm-wearable devices of FIG.12A-12C, but could also be used with other types of wearable devices forsensing neuromuscular signals (such as body-wearable or head-wearabledevices that might have neuromuscular sensors closer to the brain orspinal column).

In some embodiments, a wrist-wearable device can be used in conjunctionwith a head-wearable device described below, and the wrist-wearabledevice can also be configured to be used to allow a user to controlaspect of the artificial reality (e.g., by using EMG-based gestures tocontrol user interface objects in the artificial reality and/or byallowing a user to interact with the touchscreen on the wrist-wearabledevice to also control aspects of the artificial reality). Having thusdescribed example wrist-wearable device, attention will now be turned toexample head-wearable devices, such AR glasses and VR headsets.

Example Head-Wearable Devices

FIG. 13A shows an example AR system 1300 in accordance with someembodiments. In FIG. 13A, the AR system 1300 includes an eyewear devicewith a frame 1302 configured to hold a left display device 1306-1 and aright display device 1306-2 in front of a user's eyes. The displaydevices 1306-1 and 1306-2 may act together or independently to presentan image or series of images to a user. While the AR system 1300includes two displays, embodiments of this disclosure may be implementedin AR systems with a single near-eye display (NED) or more than twoNEDs.

In some embodiments, the AR system 1300 includes one or more sensors,such as the acoustic sensors 1304. For example, the acoustic sensors1304 can generate measurement signals in response to motion of the ARsystem 1300 and may be located on substantially any portion of the frame1302. Any one of the sensors may be a position sensor, an IMU, a depthcamera assembly, or any combination thereof. In some embodiments, the ARsystem 1300 includes more or fewer sensors than are shown in FIG. 13A.In embodiments in which the sensors include an IMU, the IMU may generatecalibration data based on measurement signals from the sensors. Examplesof the sensors include, without limitation, accelerometers, gyroscopes,magnetometers, other suitable types of sensors that detect motion,sensors used for error correction of the IMU, or some combinationthereof.

In some embodiments, the AR system 1300 includes a microphone array witha plurality of acoustic sensors 1304-1 through 1304-8, referred tocollectively as the acoustic sensors 1304. The acoustic sensors 1304 maybe transducers that detect air pressure variations induced by soundwaves. In some embodiments, each acoustic sensor 1304 is configured todetect sound and convert the detected sound into an electronic format(e.g., an analog or digital format). In some embodiments, the microphonearray includes ten acoustic sensors: 1304-1 and 1304-2 designed to beplaced inside a corresponding ear of the user, acoustic sensors 1304-3,1304-4, 1304-5, 1304-6, 1304-7, and 1304-8 positioned at variouslocations on the frame 1302, and acoustic sensors positioned on acorresponding neckband, where the neckband is an optional component ofthe system that is not present in certain embodiments of theartificial-reality systems discussed herein.

The configuration of the acoustic sensors 1304 of the microphone arraymay vary. While the AR system 1300 is shown in FIG. 13A having tenacoustic sensors 1304, the number of acoustic sensors 1304 may be moreor fewer than ten. In some situations, using more acoustic sensors 1304increases the amount of audio information collected and/or thesensitivity and accuracy of the audio information. In contrast, in somesituations, using a lower number of acoustic sensors 1304 decreases thecomputing power required by a controller to process the collected audioinformation. In addition, the position of each acoustic sensor 1304 ofthe microphone array may vary. For example, the position of an acousticsensor 1304 may include a defined position on the user, a definedcoordinate on the frame 1302, an orientation associated with eachacoustic sensor, or some combination thereof.

The acoustic sensors 1304-1 and 1304-2 may be positioned on differentparts of the user's ear. In some embodiments, there are additionalacoustic sensors on or surrounding the ear in addition to acousticsensors 1304 inside the ear canal. In some situations, having anacoustic sensor positioned next to an ear canal of a user enables themicrophone array to collect information on how sounds arrive at the earcanal. By positioning at least two of the acoustic sensors 1304 oneither side of a user's head (e.g., as binaural microphones), the ARdevice 1300 is able to simulate binaural hearing and capture a 3D stereosound field around a user's head. In some embodiments, the acousticsensors 1304-1 and 1304-2 are connected to the AR system 1300 via awired connection, and in other embodiments, the acoustic sensors 1304-1and 1304-2 are connected to the AR system 1300 via a wireless connection(e.g., a Bluetooth connection). In some embodiments, the AR system 1300does not include the acoustic sensors 1304-1 and 1304-2.

The acoustic sensors 1304 on the frame 1302 may be positioned along thelength of the temples, across the bridge of the nose, above or below thedisplay devices 1306-1 and 1306-2, or in some combination thereof. Theacoustic sensors 1304 may be oriented such that the microphone array isable to detect sounds in a wide range of directions surrounding the userthat is wearing the AR system 1300. In some embodiments, a calibrationprocess is performed during manufacturing of the AR system 1300 todetermine relative positioning of each acoustic sensor 1304 in themicrophone array.

In some embodiments, the eyewear device further includes, or iscommunicatively coupled to, an external device (e.g., a paired device),such as the optional neckband discussed above. In some embodiments, theoptional neckband is coupled to the eyewear device via one or moreconnectors. The connectors may be wired or wireless connectors and mayinclude electrical and/or non-electrical (e.g., structural) components.In some embodiments, the eyewear device and the neckband operateindependently without any wired or wireless connection between them. Insome embodiments, the components of the eyewear device and the neckbandare located on one or more additional peripheral devices paired with theeyewear device, the neckband, or some combination thereof. Furthermore,the neckband is intended to represent any suitable type or form ofpaired device. Thus, the following discussion of neckband may also applyto various other paired devices, such as smart watches, smart phones,wrist bands, other wearable devices, hand-held controllers, tabletcomputers, or laptop computers.

In some situations, pairing external devices, such as the optionalneckband, with the AR eyewear device enables the AR eyewear device toachieve the form factor of a pair of glasses while still providingsufficient battery and computation power for expanded capabilities.Some, or all, of the battery power, computational resources, and/oradditional features of the AR system 1300 may be provided by a paireddevice or shared between a paired device and an eyewear device, thusreducing the weight, heat profile, and form factor of the eyewear deviceoverall while still retaining desired functionality. For example, theneckband may allow components that would otherwise be included on aneyewear device to be included in the neckband thereby shifting a weightload from a user's head to a user's shoulders. In some embodiments, theneckband has a larger surface area over which to diffuse and disperseheat to the ambient environment. Thus, the neckband may allow forgreater battery and computation capacity than might otherwise have beenpossible on a stand-alone eyewear device. Because weight carried in theneckband may be less invasive to a user than weight carried in theeyewear device, a user may tolerate wearing a lighter eyewear device andcarrying or wearing the paired device for greater lengths of time thanthe user would tolerate wearing a heavy, stand-alone eyewear device,thereby enabling an artificial-reality environment to be incorporatedmore fully into a user's day-to-day activities.

In some embodiments, the optional neckband is communicatively coupledwith the eyewear device and/or to other devices. The other devices mayprovide certain functions (e.g., tracking, localizing, depth mapping,processing, storage, etc.) to the AR system 1300. In some embodiments,the neckband includes a controller and a power source. In someembodiments, the acoustic sensors of the neckband are configured todetect sound and convert the detected sound into an electronic format(analog or digital).

The controller of the neckband processes information generated by thesensors on the neckband and/or the AR system 1300. For example, thecontroller may process information from the acoustic sensors 1304. Foreach detected sound, the controller may perform a direction of arrival(DOA) estimation to estimate a direction from which the detected soundarrived at the microphone array. As the microphone array detects sounds,the controller may populate an audio data set with the information. Inembodiments in which the AR system 1300 includes an IMU, the controllermay compute all inertial and spatial calculations from the IMU locatedon the eyewear device. The connector may convey information between theeyewear device and the neckband and between the eyewear device and thecontroller. The information may be in the form of optical data,electrical data, wireless data, or any other transmittable data form.Moving the processing of information generated by the eyewear device tothe neckband may reduce weight and heat in the eyewear device, making itmore comfortable and safer for a user.

In some embodiments, the power source in the neckband provides power tothe eyewear device and the neckband. The power source may include,without limitation, lithium-ion batteries, lithium-polymer batteries,primary lithium batteries, alkaline batteries, or any other form ofpower storage. In some embodiments, the power source is a wired powersource.

As noted, some artificial-reality systems may, instead of blending anartificial reality with actual reality, substantially replace one ormore of a user's sensory perceptions of the real world with a virtualexperience. One example of this type of system is a head-wearabledisplay system, such as the VR system 1350 in FIG. 13B, which mostly orcompletely covers a user's field of view.

FIG. 13B shows a VR system 1350 (e.g., also referred to herein as VRheadsets or VR headset) in accordance with some embodiments. The VRsystem 1350 includes a head-mounted display (HMD) 1352. The HMD 1352includes a front body 1356 and a frame 1354 (e.g., a strap or band)shaped to fit around a user's head. In some embodiments, the HMD 1352includes output audio transducers 1358-1 and 1358-2, as shown in FIG.13B (e.g., transducers). In some embodiments, the front body 1356 and/orthe frame 1354 includes one or more electronic elements, including oneor more electronic displays, one or more IMUs, one or more trackingemitters or detectors, and/or any other suitable device or sensor forcreating an artificial-reality experience.

Artificial-reality systems may include a variety of types of visualfeedback mechanisms. For example, display devices in the AR system 1300and/or the VR system 1350 may include one or more liquid-crystaldisplays (LCDs), light emitting diode (LED) displays, organic LED (OLED)displays, and/or any other suitable type of display screen.Artificial-reality systems may include a single display screen for botheyes or may provide a display screen for each eye, which may allow foradditional flexibility for varifocal adjustments or for correcting arefractive error associated with the user's vision. Someartificial-reality systems also include optical subsystems having one ormore lenses (e.g., conventional concave or convex lenses, Fresnellenses, or adjustable liquid lenses) through which a user may view adisplay screen.

In addition to or instead of using display screens, someartificial-reality systems include one or more projection systems. Forexample, display devices in the AR system 1300 and/or the VR system 1350may include micro-LED projectors that project light (e.g., using awaveguide) into display devices, such as clear combiner lenses thatallow ambient light to pass through. The display devices may refract theprojected light toward a user's pupil and may enable a user tosimultaneously view both artificial-reality content and the real world.Artificial-reality systems may also be configured with any othersuitable type or form of image projection system.

Artificial-reality systems may also include various types of computervision components and subsystems. For example, the AR system 1300 and/orthe VR system 1350 can include one or more optical sensors such astwo-dimensional (2D) or three-dimensional (3D) cameras, time-of-flightdepth sensors, single-beam or sweeping laser rangefinders, 3D LiDARsensors, and/or any other suitable type or form of optical sensor. Anartificial-reality system may process data from one or more of thesesensors to identify a location of a user, to map the real world, toprovide a user with context about real-world surroundings, and/or toperform a variety of other functions. For example, FIG. 13B shows VRsystem 1350 having cameras 1360-1 and 1360-2 that can be used to providedepth information for creating a voxel field and a two-dimensional meshto provide object information to the user to avoid collisions. FIG. 13Balso shows that the VR system includes one or more additional cameras1362 that are configured to augment the cameras 1360-1 and 1360-2 byproviding more information. For example, the additional cameras 1362 canbe used to supply color information that is not discerned by cameras1360-1 and 1360-2. In some embodiments, cameras 1360-1 and 1360-2 andadditional cameras 1362 can include an optional IR cut filter configuredto remove IR light from being received at the respective camera sensors.

In some embodiments, the AR system 1300 and/or the VR system 1350 caninclude haptic (tactile) feedback systems, which may be incorporatedinto headwear, gloves, body suits, handheld controllers, environmentaldevices (e.g., chairs or floormats), and/or any other type of device orsystem, such as the wearable devices discussed herein. The hapticfeedback systems may provide various types of cutaneous feedback,including vibration, force, traction, shear, texture, and/ortemperature. The haptic feedback systems may also provide various typesof kinesthetic feedback, such as motion and compliance. The hapticfeedback may be implemented using motors, piezoelectric actuators,fluidic systems, and/or a variety of other types of feedback mechanisms.The haptic feedback systems may be implemented independently of otherartificial-reality devices, within other artificial-reality devices,and/or in conjunction with other artificial-reality devices.

The techniques described above can be used with any device forinteracting with an artificial-reality environment, including thehead-wearable devices of FIG. 13A-13B, but could also be used with othertypes of wearable devices for sensing neuromuscular signals (such asbody-wearable or head-wearable devices that might have neuromuscularsensors closer to the brain or spinal column). Having thus describedexample wrist-wearable device and head-wearable devices, attention willnow be turned to example feedback systems that can be integrated intothe devices described above or be a separate device.

Having thus described system-block diagrams and then example wearabledevices, attention will now be directed to certain example embodiments.

Example Aspects

A few example aspects will now be briefly described.

-   -   (A1) In accordance with some embodiments, a method of using a        gesture detected based on data from a wrist-wearable device to        cause presentation of camera data at a head-wearable device is        disclosed. The method includes receiving data generated based on        performance of an in-air hand gesture by a user is wearing a        wrist-wearable device that is wearing a wrist-wearable device        and, in accordance with a determination that the data indicates        that the in-air hand gesture is a camera-control gesture,        causing presentation to the user, via a display of a        head-wearable device that is worn by the user and that is in        communication with the wrist-wearable device, of a        representation of camera data from a camera that is in        communication with one or both of the wrist-wearable device and        the head-wearable device.    -   (A2) In some embodiments of A1, the camera is selected from        among one or more available cameras that are communicatively        coupled to the wrist-wearable device or the head-wearable device        based on a characteristic of the in-air hand gesture.    -   (A3) In some embodiments of any of A1 and A2, the camera-control        gesture causes capturing of an image using the camera, and the        causing presentation to the user includes causing presentation,        via the display of the head-wearable device, of the image        captured by the camera.    -   (A4) In some embodiments of A3, the method further includes,        after the image is captured, causing presentation of one or more        options for sharing the image, and responsive to a user        selection of a first option of the one or more options for        sharing the image, causing the image to be sent to another        electronic device.    -   (A5) In some embodiments of A4, the method further includes        responsive to a user selection of a second option, distinct from        the first option, of the one or more options for sharing the        image with a specific contact, causing the image to be send to        an electronic device associated with the specific contact.    -   (A6) In some embodiments of any of A3-A5, the image is one of a        plurality of images, each respective image caused to be captured        in accordance with the determination that the data indicates        that the in-air hand gesture is the camera-control gesture, and        the causing presentation to the user includes causing        presentation of respective representations of each respective        image of the plurality of images.    -   (A7) In some embodiments of any of A1-A6, the data is generated        using, in part, one or more sensors of the wrist-wearable        device, and while data generated by the one or more sensors of        the wrist-wearable device indicates that the in-air hand gesture        is maintained, continuing to cause the presentation of the        representation of the camera data via the display of the        head-wearable device.    -   (A8) In some embodiments of A7, continuing to cause the        presentation of the representation of the camera data via the        display of the head-wearable device includes causing        presentation of a representation of a field of view of the        camera to the user via the display of the head-wearable device.    -   (A9) In some embodiments of any of A7-A8, the method further        includes in accordance with a determination that data generated        by the one or more sensors of the wrist-wearable device        indicates that the in-air hand gesture is no longer maintained,        causing the presentation of the representation of the camera        data via the display of the head-wearable device to cease.    -   (A10) In some embodiments of any of A7-A9, the method further        includes, in accordance with a determination that data generated        by the one or more sensors of the wrist-wearable device        indicates that the in-air hand gesture is no longer maintained,        causing a capture of an image within the field of view using the        camera, and causing presentation of a representation of the        image via the display of the head-wearable device.    -   (A11) In some embodiments of any of A7-A10, the in-air hand        gesture is a single pinch gesture performed using at least two        phalanges of a user's hand contacting one another, without any        contact with a display of the wrist-wearable device.    -   (A12) In some embodiments of any of A1-A11, the method further        includes receiving additional data generated based on        performance of an additional in-air hand gesture by the user        while the user is wearing the wrist-wearable device and, in        accordance with a determination that the additional data        indicates that the additional in-air hand gesture is not a        camera-control gesture, forgo causing presentation of camera        data via the display of the head-wearable device and causing        activation of a microphone of the head-wearable device for        allowing the user to provide a voice command to a digital        assistant. The microphone is in communication with one or both        of the wrist-wearable device and the head-wearable device.    -   (A13) In some embodiments of any of A1-A12, the in-air hand        gesture is a first in-air hand gesture, and the method further        includes, responsive to a determination that a second in-air        hand gesture is performed while the first in-air hand gesture is        maintained, causing a zoomed-in representation of the camera        data to be presented via the display of the head-wearable        device.    -   (A14) In some embodiments of A13, the method further includes in        accordance with a determination that the first in-air hand        gesture is no longer maintained while the zoomed-in        representation of the camera data is being presented via the        display of the head-wearable device, causing the camera to        capture the zoomed-in representation of the camera data and        causing presentation to the user, via the display of the        head-wearable device that is worn by the user, an image of the        zoomed-in representation of the camera data.    -   (A15) In some embodiments of A13, the method further includes        responsive to a determination that a third in-air hand gesture        is performed while the first in-air hand gesture is maintained,        causing a zoomed-out representation of the camera data to be        presented via the display of the head-wearable device.    -   (A16) In some embodiments of A15, the method includes, in        accordance with a determination that the first in-air hand        gesture is no longer maintained while the zoomed-out        representation of the camera data is being presented via the        display of the head-wearable device, causing the camera to        capture the zoomed-out representation of the camera data and        causing presentation to the user, via the display of the        head-wearable device that is worn by the user, an image of the        zoomed-out representation of the camera data.    -   (A17) In some embodiments of A15 or A16, the first in-air hand        gesture is a maintained contact between at least two phalanges        of a user's hand, the second in-air hand gesture is an in-air        rotational movement of the user's wrist in a first direction,        and the third in-air hand gesture is an in-air rotational        movement of the user's wrist in a second direction distinct from        the first direction.    -   (A18) In some embodiments of A12, the method further includes,        upon determining that further data indicates that the additional        in-air hand gesture is no longer maintained, causing        deactivation of the microphone that is in communication with one        or both of the wrist-wearable device and the head-wearable        device.    -   (A19) In some embodiments of any of A1-A18, the camera is a        first camera, and the method further includes in accordance with        a determination that one or more characteristics of the in-air        hand gesture are within a first threshold, causing the first        camera to capture the camera data for presentation at the        display of the head-wearable device that is worn by the user.        The method also includes, in accordance with a determination        that the one or more characteristics of the in-air hand gesture        are within a second threshold, causing a second camera to        capture the camera data for presentation at the display of the        head-wearable device that is worn by the user.    -   (A20) In some embodiments of A19, the first camera is in        proximity to the user and the second camera is remote to the        user    -   (A21) In some embodiments of any of A19 and A20, the one or more        characteristics of the in-air hand gesture include one or more        of a force, a duration, a number of contacts, and a pattern.    -   (A22) In some embodiments of A1, the method further includes        receiving, via one or more sensors of the head-wearable device,        facial movement related data based on the performance of a        vestigial ear muscle gesture by the user that is wearing the        head-wearable device. The method includes, in accordance with a        determination that the facial movement related data indicates        that the vestigial ear muscle gesture is a control gesture,        causing the head-wearable device to provide one or more        instructions for performing the control gesture at the        head-wearable device and/or another communicatively coupled        device.    -   (A23) In some embodiments of A22, the method further includes in        accordance with a determination that the vestigial ear muscle        gesture is a camera-control gesture for capturing image data via        a communicatively coupled imaging device, capturing camera data        via a camera that is in communicatively coupled with the        head-wearable device and presenting to the user, via the display        of the head-wearable device of a representation of camera data        from the camera that is in communication with the head-wearable        device.    -   (A24) In some embodiments of A22, the method further includes,        in accordance with a determination that the vestigial ear muscle        gesture is a video-control gesture, capturing a video using the        camera.    -   (A25) In some embodiments of A22, the method further includes,        in accordance with a determination that the vestigial ear muscle        gesture is an application control-gesture, causing performance        of one or more application specific operations.    -   (B1) In accordance with some embodiments, a method of being        responsive to an indication that a user wearing a head-wearable        device requests to enable vestigial ear muscle detection,        initiating a vestigial muscle training mode. The method includes        wile in vestigial training mode, presenting instructions to the        user to perform a facial movement to move a respective vestigial        ear muscle of one or more vestigial ear muscles. The method        further includes in accordance with a determination that facial        movement related data detected via sensors of the head-wearable        device and based on the facial movement performed by the user        satisfies detectable vestigial ear muscle movement criteria        associated with movement of the respective vestigial ear muscle,        providing an indication that a vestigial ear muscle movement was        detected, receiving a user input to assign a vestigial ear        muscle gesture to the vestigial ear muscle movement, and storing        the facial movement related data associated with the vestigial        ear muscle gesture.    -   (B2) In some embodiments of B1, the method includes in        accordance with a determination that the facial movement related        data does not satisfy the detectable vestigial ear muscle        movement criteria associated with movement of the respective        vestigial ear muscle, providing an indication that the        respective vestigial ear muscle movement was not detected, and        presenting additional instructions to the user to perform an        additional facial movement.    -   (B3) In some embodiments of B2, presenting the additional        instructions to the user to perform the facial movement includes        instructing the user to move one or more facial muscles        connected to the one or more vestigial ear muscles.    -   (B4) In some embodiments of B3, the method further includes in        accordance with a determination that updated facial movement        related data detected based on the movement of the facial muscle        connected to the respective vestigial ear muscle satisfies        detectable vestigial ear muscle movement criteria associated        with movement of the respective vestigial ear muscle providing        an indication that the vestigial ear muscle movement was        detected, and presenting further instructions to the user to        perform a further additional facial movement to isolate the one        or more vestigial ear muscles without movement of the facial        muscle connected to the respective vestigial ear muscle.    -   (B5) In some embodiments of any of B1-B4, the facial movement is        a first facial movement, the facial movement related data is        first facial movement related data, and the vestigial ear muscle        gesture is a first vestigial ear muscle gesture. The method        further includes after storing the first facial movement related        data associated with the first vestigial ear muscle gesture,        presenting instructions to the user to perform a second facial        movement to move the respective vestigial ear muscle of the one        or more vestigial ear muscles in a first direction. The method        also includes, in accordance with a determination that second        facial movement related data detected based on the second facial        movement performed by the user satisfies detectable vestigial        ear muscle movement criteria associated with movement of the        respective vestigial ear muscle in the first direction,        providing an indication that a first directional vestigial ear        muscle movement was detected, receiving a user input to assign a        second vestigial ear muscle gesture to the first directional        vestigial ear muscle movement, and storing the second facial        movement related data associated with the second vestigial ear        muscle gesture.    -   (B6) In some embodiments of B1-B5, the facial movement is a        first facial movement, the facial movement related data is first        facial movement related data, and the vestigial ear muscle        gesture is a first vestigial ear muscle gesture. The method        further comprises, after storing the first facial movement        related data associated with the first vestigial ear muscle        gesture, presenting instructions to the user to perform a third        facial movement to move a plurality of the one or more vestigial        ear muscles. The method also includes in accordance with a        determination that third facial movement related data detected        based on the third facial movement performed by the user        satisfies detectable vestigial ear muscle movement criteria        associated with movement of the plurality of the one or more        vestigial ear muscles, providing an indication that a plurality        of vestigial ear muscle movements were detected, receiving a        user input to assign a third vestigial ear muscle gesture to the        plurality of vestigial ear muscle movements, and storing the        third facial movement related data associated with the third        vestigial ear muscle gesture.    -   (B7) In some embodiments of B6, wherein the first facial        movement corresponds to movement of a first ear.    -   (B8) In some embodiments of B7, wherein the third facial        movement to move the plurality of the one or more vestigial ear        muscles correspond to moving the first ear and a second ear.    -   (B9) In some embodiments of B1-B8, wherein the indication is at        least one of a visual indication, auditory indication, or haptic        feedback indication.    -   (B10) In some embodiments of B1-B9, wherein the sensors of the        head-wearable device are one or more EMG sensors.    -   (C1) In accordance with some embodiments, a head-wearable device        for capturing and presenting representations of camera data        responsive to a gesture performed at a wrist-wearable device,        the head-wearable device configured to perform or cause        performance of the method of any of A1-B10.    -   (D1) In accordance with some embodiments, a system for using a        gesture detected at a wrist-wearable device to cause        presentation of camera data at a head-wearable device, the        system configured to perform or cause performance of the method        of any of A1-B10.    -   (E1) In accordance with some embodiments, non-transitory,        computer-readable storage medium including instructions that,        when executed by a head-wearable device, a wrist-wearable        device, or an intermediary device in communication with the        head-wearable device and the wrist-wearable device, cause        performance of the method of any of A1-B10.    -   (F1) In another aspect, a means on a wrist-wearable device,        head-wearable device, and/or intermediary device for performing        or causing performance of the method of any of claims A1-B10.    -   (G1) In accordance with some embodiments, a wrist-wearable        device for detecting a gesture and causing presentation of        camera data at a head-wearable device, the wrist-wearable device        configured to perform or cause performance of the method of any        of claims A1-B10.

It will be understood that, although the terms “first,” “second,” etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the claims. Asused in the description of the embodiments and the appended claims, thesingular forms “a,” “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willalso be understood that the term “and/or” as used herein refers to andencompasses any and all possible combinations of one or more of theassociated listed items. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

As used herein, the term “if” can be construed to mean “when” or “upon”or “in response to determining” or “in accordance with a determination”or “in response to detecting,” that a stated condition precedent istrue, depending on the context. Similarly, the phrase “if it isdetermined [that a stated condition precedent is true]” or “if [a statedcondition precedent is true]” or “when [a stated condition precedent istrue]” can be construed to mean “upon determining” or “in response todetermining” or “in accordance with a determination” or “upon detecting”or “in response to detecting” that the stated condition precedent istrue, depending on the context.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the claims to the precise forms disclosed. Many modifications andvariations are possible in view of the above teachings. The embodimentswere chosen and described in order to best explain principles ofoperation and practical applications, to thereby enable others skilledin the art.

What is claimed is:
 1. A method of using a gesture detected based ondata from a wrist-wearable device to cause presentation of camera dataat a head-wearable device, the method comprising: receiving datagenerated based on performance of an in-air hand gesture by a user thatis wearing the wrist-wearable device, the data being received from thewrist-wearable device; and in accordance with a determination that thedata indicates that the in-air hand gesture is a camera-control gesture:causing presentation to the user, via a display of a head-wearabledevice that is worn by the user and that is in communication with thewrist-wearable device, of a representation of camera data from a camerathat is in communication with one or both of the wrist-wearable deviceand the head-wearable device.
 2. The method of claim 1, wherein: thecamera is selected from among one or more available cameras that arecommunicatively coupled to the wrist-wearable device or thehead-wearable device based on a characteristic of the in-air handgesture.
 3. The method of any of claim 1, wherein: the camera-controlgesture causes capturing of an image using the camera, and the causingpresentation to the user includes causing presentation, via the displayof the head-wearable device, of the image captured by the camera.
 4. Themethod of claim 3, further comprising: after the image is captured,causing presentation of one or more options for sharing the image; andresponsive to a user selection of a first option of the one or moreoptions for sharing the image, causing the image to be sent to anotherelectronic device.
 5. The method of claim 4, further comprising:responsive to a user selection of a second option, distinct from thefirst option, of the one or more options for sharing the image with aspecific contact, causing the image to be sent to an electronic deviceassociated with the specific contact.
 6. The method of any of claim 3,wherein: the image is one of a plurality of images, each respectiveimage caused to be captured in accordance with the determination thatthe data indicates that the in-air hand gesture is the camera-controlgesture, and the causing presentation to the user includes causingpresentation of respective representations of each respective image ofthe plurality of images.
 7. The method of any of claim 1, wherein: thedata is generated using, in part, one or more sensors of thewrist-wearable device, and while data generated by the one or moresensors of the wrist-wearable device indicates that the in-air handgesture is maintained, continuing to cause the presentation of therepresentation of the camera data via the display of the head-wearabledevice including causing presentation of a representation of a field ofview of the camera to the user via the display of the head-wearabledevice.
 8. The method of any of claim 7, further comprising: inaccordance with a determination that data generated by the one or moresensors of the wrist-wearable device indicates that the in-air handgesture is no longer maintained, causing the presentation of therepresentation of the camera data via the display of the head-wearabledevice to cease.
 9. The method of any of claim 7, further comprising: inaccordance with a determination that data generated by the one or moresensors of the wrist-wearable device indicates that the in-air handgesture is no longer maintained: causing a capture of an image withinthe field of view using the camera, and causing presentation of arepresentation of the image via the display of the head-wearable device.10. The method of any of claim 7, wherein the in-air hand gesture is asingle pinch gesture performed using at least two phalanges of a user'shand contacting one another, without any contact with a display of thewrist-wearable device.
 11. The method of any of claim 1, furthercomprising: receiving additional data generated based on performance ofan additional in-air hand gesture by the user while the user is wearingthe wrist-wearable device; and in accordance with a determination thatthe additional data indicates that the additional in-air hand gesture isnot a camera-control gesture, forgo causing presentation of camera datavia the display of the head-wearable device and causing activation of amicrophone for allowing the user to provide a voice command to a digitalassistant, wherein the microphone is in communication with one or bothof the wrist-wearable device and the head-wearable device.
 12. Themethod of any of claim 1, wherein the in-air hand gesture is a firstin-air hand gesture, and the method further comprises: responsive to adetermination that a second in-air hand gesture is performed while thefirst in-air hand gesture is maintained, causing a zoomed-inrepresentation of the camera data to be presented via the display of thehead-wearable device.
 13. The method of any of claim 12, furthercomprising: in accordance with a determination that the first in-airhand gesture is no longer maintained while the zoomed-in representationof the camera data is being presented via the display of thehead-wearable device, causing the camera to capture the zoomed-inrepresentation of the camera data; and causing presentation to the user,via the display of the head-wearable device that is worn by the user, animage of the zoomed-in representation of the camera data.
 14. The methodof claim 12, further comprising: responsive to a determination that athird in-air hand gesture is performed while the first in-air handgesture is maintained, causing a zoomed-out representation of the cameradata to be presented via the display of the head-wearable device. 15.The method of claim 14, further comprising: in accordance with adetermination that the first in-air hand gesture is no longer maintainedwhile the zoomed-out representation of the camera data is beingpresented via the display of the head-wearable device, causing thecamera to capture the zoomed-out representation of the camera data; andcausing presentation to the user, via the display of the head-wearabledevice that is worn by the user, of an image of the zoomed-outrepresentation of the camera data.
 16. The method of claim 14, wherein:the first in-air hand gesture is a maintained contact between at leasttwo phalanges of a user's hand; the second in-air hand gesture is anin-air rotational movement of the user's wrist in a first direction; andthe third in-air hand gesture is an in-air rotational movement of theuser's wrist in a second direction distinct from the first direction.17. The method of claim 1: wherein the camera is a first camera, and themethod further comprises: in accordance with a determination that one ormore characteristics of the in-air hand gesture are within a firstthreshold, causing the first camera to capture the camera data forpresentation at the display of the head-wearable device that is worn bythe user; and in accordance with a determination that the one or morecharacteristics of the in-air hand gesture are within a secondthreshold, causing a second camera to capture the camera data forpresentation at the display of the head-wearable device that is worn bythe user.
 18. The method of claim 1, further comprising: receiving, viaone or more sensors of the head-wearable device, facial movement relateddata based on performance of a vestigial ear muscle gesture by the userthat is wearing the head-wearable device; and in accordance with adetermination that the facial movement related data indicates that thevestigial ear muscle gesture is a control gesture, causing thehead-wearable device to provide one or more instructions for performingthe control gesture at the head-wearable device and/or anothercommunicatively coupled device.
 19. A system comprising a head-wearabledevice and a wrist-wearable device, the system configured to: receivedata generated based on performance of an in-air hand gesture by a userthat is wearing the wrist-wearable device, the data being received fromthe wrist-wearable device; and in accordance with a determination thatthe data indicates that the in-air hand gesture is a camera-controlgesture: causing presentation to the user, via a display of ahead-wearable device that is worn by the user and that is incommunication with the wrist-wearable device, of a representation ofcamera data from a camera that is in communication with one or both ofthe wrist-wearable device and the head-wearable device.
 20. Anon-transitory, computer-readable storage medium including instructionsthat when executed by a system including a head-wearable device and awrist-wearable device, cause the system to: receive data generated basedon performance of an in-air hand gesture by a user that is wearing thewrist-wearable device, the data being received from the wrist-wearabledevice; and in accordance with a determination that the data indicatesthat the in-air hand gesture is a camera-control gesture: causepresentation to the user, via a display of a head-wearable device thatis worn by the user and that is in communication with the wrist-wearabledevice, of a representation of camera data from a camera that is incommunication with one or both of the wrist-wearable device and thehead-wearable device.